diff options
Diffstat (limited to 'bindings/cpp/doc/stream_processing.rst')
-rw-r--r-- | bindings/cpp/doc/stream_processing.rst | 48 |
1 files changed, 48 insertions, 0 deletions
diff --git a/bindings/cpp/doc/stream_processing.rst b/bindings/cpp/doc/stream_processing.rst new file mode 100644 index 00000000..39265287 --- /dev/null +++ b/bindings/cpp/doc/stream_processing.rst @@ -0,0 +1,48 @@ +Stream Processing +================= + +.. default-domain:: cpp +.. highlight:: cpp +.. namespace:: serd + +The above examples show how a document can be either written to a file or loaded into a model, +simply by changing the sink that the data is written to. +There are also sinks that filter or transform the data before passing it on to another sink, +which can be used to build more advanced pipelines with several processing stages. + +Canonical Literals +------------------ + +A `canon` is a stream processor that converts literals with supported XSD datatypes into canonical form. +For example, this will rewrite an xsd:decimal literal like ".10" as "0.1". +A canon can be constructed by passing the "target" sink that the transformed statements should be written to, +for example: + +.. literalinclude:: overview.cpp + :start-after: begin canon-new + :end-before: end canon-new + :dedent: 2 + +The last argument is a bitwise ``OR`` of :enum:`CanonFlag` flags. +For example, :enumerator:`CanonFlag::lax` will tolerate and pass through invalid literals, +which can be useful for cleaning up questionabe data as much as possible without losing any information. + +Filtering Statements +-------------------- + +A `filter` is a stream processor that filters statements based on a pattern. +It can be configured in either inclusive or exclusive mode, +which passes through only statements that match or don't match the pattern, +respectively. +A filter can be constructed by passing the target sink, +the statement pattern as individual nodes, +and an inclusive flag. +For example, all statements with predicate ``rdf:type`` could be filtered out when loading a model: + +.. literalinclude:: overview.cpp + :start-after: begin filter-new + :end-before: end filter-new + :dedent: 2 + +If ``false`` is passed for the last parameter instead, +then the filter operates in exclusive mode and will instead insert only statements with predicate ``rdf:type``. |