Advanced SPARQL Lecture

Public

By VU University Amsterdam (Vrije Universiteit)

Go to https://triplydb.com/vu and click on this story :-)

My name is Wouter Beek, founder of Triply. We are always eager to hire new talent or support-supervise BSc- and MSc-theses. We lead a large Digital Product Passports project with a large research component. So feel free to contact met at wouter@triply.cc We are located at the VU campus, so easy to reach.

In today's lecture we will dive a bit deeper into the interestingly innovative world of SPARQL querying, and its many applications in industry and government.

Remember the SPARQL basics

Feature	Keyword
Tabular result format	`Select *`
Triple Pattern	`?s <p> <o>; ?q 'r', [].`
Namespace	`Prefix alias: <...>`
Information that may be absent	`Optional { ... }`
Exclude results based on values	`Filter(... && ... \|\| ... )`
Functions that calculate new values	`langMatches(..., ...)`, `lang(...)`, etc.
Introduce a value that does not (directly) come from the graph	`Bind(... as ?x)`
Datatyped value	`'...'^^rdf:HTML`
Ordered results	`Order By Desc(?a) ... Asc(?z)`
Limited result set	`Offset M`, `Limit N`

go to Dataset Try this query yourself

Some new keywords

Feature	Keyword
Aggregate	`Group By ?a ... ?z`, `(Sample(...) as ?x)`
Conditional evaluation	`If(..., ..., ...)`
Casting functions	`xsd:gYear(...)`
More functions	`Concat(...)`, `Str(...)`, `Strdt(..., ...)`

Learn more about aggregates
Learn more about GeoSPARQL

go to Dataset Try this query yourself

Extreme Federation

Normal federative query: some content is retrieved from the local database + some extra content is retrieved from a remote URL
Extreme federative query: all content is retrieved from a remote URL

Properties of Extreme Federation:

Evaluation Guarantee: Query results are independent of the location where the query is evaluated.
Standardized: Query results only depend on (1) the authority of the data source, (2) the standardization of the used vocabalaries, (3) the standards-conformity of the evaluation context.
Decentralization: Data is published in a highly decentralized way (thousands of endpoints).

	Data	Domain knowledge	Evaluation Guarantee
Traditional Data Paradigms	Centralized (query within a silo)	Centralized (schema only meaningful inside a silo)	Dependent on the one organization that runs the centralized database
Extreme Federation	Decentralized (query the Internet of endpoints)	Decentralized (based on shared vocabularies)	Every organization can run the query anywhere on the Internet (while respecting access rights etc.)

Outline:

select * {
  # Retrieve some data from authoritative source A.
  service <...> {
    ...
  }
  # Retrieve some data from authoritative source B.
  service <...> {
    ...
  }
  # Perform operations that are source-indepent.
  filter(...)
  bind(...)
}

Example:

RCE: https://api.linkeddata.cultureelerfgoed.nl/s/zRHMGuu1y
Kadaster: https://api.labs.kadaster.nl/s/1ULLENPtp

SPARQL functions

See for more information the SPARQL Functions section in the SHACL standard.

SPARQL queries can get quite complex. It is useful to write and maintain modular components of SPARQL as separate functions.

For example, querying things that have predefined order -- like a list of steps in a recipe -- is a bit awkward in native SPARQL:

go to Dataset Try this query yourself

Luckily, it is possible to isolate the complex logic for dealing with the order of items in separate functions. For example:

sparql:list_nth1
  a sh:SPARQLFunction;
  sh:parameter
    [ a sh:Parameter;
      sh:class rdf:List;
      sh:nodeKind sh:BlankNodeOrIRI;
      sh:order 1;
      sh:path param:list ],
    [ a sh:Parameter;
      sh:datatype xsd:integer;
      sh:nodeKind sh:Literal;
      sh:order 2 ];
  sh:path param:index.
  sh:returnType rdfs:Resource;
  sh:select '''
    select $return {
      $list rdf:rest* ?sublist.
      ?sublist rdf:rest*/rdf:first $return.
    }
    group by $return
    having(count(?sublist) = $index)'''.

We can use this function to retrieve items based on their index in a list:

go to Dataset Try this query yourself

As another example of a SPARQL function, we can return the index of a given item in a given list (notice that the same item could -- in theory -- be at different indices in different lists):

sparql:list_index
  a sh:SPARQLFunction;
  sh:parameter
    [ sh:class rdf:List;
      sh:nodeKind sh:BlankNodeOrIRI;
      sh:order 1;
      sh:path param:list ],
    [ sh:class rdfs:Resource;
      sh:order 2;
      sh:path param:item.
      sh:returnType xsd:integer ];
  sh:select '''
    select (count(?sublist) as $return) {
      $list rdf:rest* ?sublist.
      ?sublist rdf:rest*/rdf:first $item.
    }
    having (count(?sublist) > 0)'''.

With this function in place, we can put the steps in the recipe in order:

go to Dataset Try this query yourself

SPARQL Functions are radically innovative.

The innovative part is not: the fact that I can store a piece of algorithmic content for later execution through a function call. After all, any programming can do this; SQL functions can do this; etc.

The innovative part is: the fact that I can add new function as linked data to my own store, and thereby make reusable functionality available to anybody who queries that store. Specifically, I do not depend on the SPARQL engine developers to include some specific function. I can add this function myself, as a data scientist.

Also, collections of commonly used SPARQL Functions can be interchanged with others in a portable way. As a result, domain-specific functions can be included in domain vocabulary. In a vocabulary about building permits, we include SPARQL Functions that calculation (part of) the automated building controls.

In addition to functions, the SHACL standard also included SPARQL Rules, that can be executed in a rules engine. (We will not cover rules today.)

Create a RESTful API with SPARQL

go to Dataset Try this query yourself

Source datasets:

Empty dataset

SPARQL Functions