Monday, December 24, 2012

New build available! Tons of new features!

We are happy to announce the availability of a new build 2039 of -ontop- v1.7-alpha, a Java framework to query RDFS/OWL2QL ontologies with mappings to databases. The SPARQL engine, Quest, implements the latests techniques in SPARQL-to-SQL rewriting, produces very fast SQL queries and supports inference out of the box.

The framework is still in alpha and has several rough edges, however, given the performance it is currently able to deliver, and that many features are already stable, we think it might be interesting to give it a test drive. Please help us improve the stability of the build and bring it to beta grade and report any issues you encounter, we'll take all your feedback into account for the next release.

This upgrade includes bug fixes, major improvements to existing features and many new and exciting features. Some of the highlights of this build include:

SPARQL 1.0 support:

  • Support for most SPARQL built in functions, e.g., arithmetics, type checking, etc.
  • Support for CONSTRUCT and DESCRIBE result forms.
  • Support for OPTIONAL and bound, which allow you to have NEGATION in the usual SPARQL 1.0 style.
  • Support for variables in ANY position, e.g., to allow queries such as SELECT ?p ?v WHERE { ?x :name "John Smith". ?x ?p ?v }
  • Stronger compliance to standard RDF and SPARQL semantics for FILTER expressions.

Performance

  • Multiple improvements to the SPARQL-to-rule translation, mapping and TBox optimization algorithms.
  • New algorithm for query rewriting w.r.t. the TBox that increases the performance of query answering by many orders of magnitude when inference w.r.t. to existential constants is enabled. The implementation has been contributed by Dr. Roman Kontchakov.
  • Benchmarks. We finally started work on benchmarking. We tested against Virtuoso triple store, Virtuoso RDF views and D2R Server using the BSBM and FishMark benchmarks. The results show that Quest's query answering technique for 'virtual RDF graphs' can outperform these systems, in some cases by several orders of magnitude even when using an open source database as the backend. 

APIs and Accessibility

  • Completely new Sesame API provider, now users of Sesame can create "Virtual RDF repositories" with -ontop-.
  • SPARQL end-point support through integration with Sesame's workbench.

Mapping language

  • Improved support for default literal types, now it conforms with the R2RML "Natural Mapping" specification.
  • Support for language tags obtained from the DB
  • Proper support for constants in the mappings
  • Proper internal handling of NULL values in the database (as recommended by the R2RML standard)

Database support

  • Restored support for Oracle, DB2 and SQL Server
  • Improved support and performance over MySQL

Documentation

  • Revised and extended all technical documentation

The new build can be downloaded from our brand new website:

     http://ontop.inf.unibz.it/


Happy new year!
The -ontop- Team

Tuesday, November 13, 2012

Optique starts!

Last week we were in Oslo for the kick-off meeting of Optique. For those that don't know yet, Optique [1] is the EU project that will be funding research for and the development of -ontop- for the next 4 years.
The objective of the project (in simpler words that those used in the brochures) is to use semantic technology, with focus on reasoning and OBDA, to create an integrated solution that companies can use to regain control of their data. The keyword here is Big Data, big in volume of course, but also in diversity and speed.

The basic idea is that RDF, Ontology and OBDA can allow companies to perform complex data integration tasks in cheaper ways and through methodologies that will allow them to continue growing in a controlled way.

The consoritium is made by a great bunch of bright people, which are very much eager to deliver the promised results.

On the academic side the consortium includes of course the Free University  of Bolzano, but also University of Athens, Oxford University, Technical University of Hamburg, University of Oslo and Sapienza University of Rome.

On the industrial side things are also very exciting. Working on the integration of results and making a real product out of the things academic do we will have FuildOps, and specialised in the Statoil use case we will also have DNV. And something very exciting about the industry side is that we will have Siemens and Statoil too as providers for use cases, consumers and evaluators of the technology developed within the project. In both cases we are talking abouta huge number of data sources and ridiculous volumes of data.

What is extremely exciting about all this is that -ontop- and Quest are going to be central in achieving all this goals, we are all betting on efficient query rewriting and efficient execution. Because of that, we are getting funding of course and thanks to that we already grew the team from 2 people actively developing -ontop- to 4 people, and we will grow to 5 or 6 very soon. More over, we started already focusing a lot in things related to usability in order to be able to integrate with the tools from FluidsOp and to be ready to start providing results in the first months. Things are moving fast, and will move even faster in 2013.

In conclusion,  things are going to get really exciting in the OBDA are, so stay tuned!

[1] http://optique.project.ifi.uio.no/ (temporal website)

Monday, November 12, 2012

ISWC Demo tomorrow

Hi all,

Tomorrow we have a demo during ISWC (6:30 pm). If you want to take a look at what Quest can do in "virtual RDF" mode, come and visit us! I'll also talk about the techniques we use, the performance we get and the plans for the future.

Don't forget to vote for us!

- Mariano

Sunday, October 14, 2012

Status update

A small status update.

The SPARQL 1.0 implementation is around 80% done, missing still support for some of the corner cases in nested OPTIONALs with references to upper level variables. The nice part is that Josef is almost done with all the core SPARQL functions, leaving little more to be done. Timea is also preparing the Sesame provider layer for Quest that will let us offer access to Quest through Sesame's console to setup a proper SPARQL end-point.

Also, even though we haven't fully finished the implementation it is already in a good state to cover most queries, so Mindaugas is setting up the environment for running the Berlin Benchmark. Expect some number comparing MySQL D2RQ, Virtuoso virtual RDF graphs and Quest (running on top of MySQL) very very soon.

Talking about benchmarks, the paper by Bail et al. [1] was accepted. They compare MySQL, Virtuoso virtual graphs and Quest in virtual RDF mode. Some great numbers for Quest, up to 500 times faster than Virtuoso, almost as fast as native MySQL. It seems that Virtuoso virtual graphs performance drops dramatically with each join you add to the queries, 4 queries already set Virtuoso to 20 or less queries per second (qps) when Quest and MySQL are on the hundreds (500-900). Five joins and we go down to 1 qps with virtuoso, with 550 qps for Quest and 840 qps for MySQL. The paper is available here [2].

Exciting times!

[1] Samantha Bail, Sandra Alkiviadous, Bijan Parsia, David Workman, Mark van Harmelen, Rafael S. Gonçalves, and Cristina Garilao. FishMark: A Linked Data Application Benchmark. SSWS, 2012. ISWC 2012 workshop. To appear.
[2] http://owl.cs.manchester.ac.uk/fishmark/

Tuesday, September 25, 2012

DAWG Tests for SPARQL 1.0

So, we finally finished the implementation of SPARQL 1.0! There is still some way to go, many edge cases need to be sorted out. However, it will be only 1 or 2 months and we will the first version fully compatible with SPARQL 1.0.

We are also already testing the Sesame Repository implementation, so expect some huge jump in functionality very very soon ;).


Sunday, September 9, 2012

Teaser

Small teaser on the next build:

  • First build with support for all the core features of SPARQL 1.0!! (unbound predicates, e.g.,  ?x rdf:type ?c, OPTIONAL, unbound variables through UNION, related filters: bound(), isIRI(), isLiteral())!
  • Our first mapping assistant! Yes! at last some help in writing mappings in -ontopPro- (our plugin for Protege)! 
  • Proper SQL null handling
  • And a paper to describe the whole approach ;-)

Stay tunned! The next release is schedule in 5 weeks from today. If you can't wait, contact us to help us test and debug :D.

Sunday, August 26, 2012

See you in Boston for ISWC 2012!

The demo for ISWC went through! We will be having a stand during the demo session of ISWC 2012 in Boston!

This will be a great opportunity to have discussion on OBDA, mappings and projects. If you would like to meet during that week, let us know! 

Wednesday, August 22, 2012

QuestOWL is now available



We are happy to announce the availability of the first public version of QuestOWL, a version of Quest that is compatible with the OWLAPI 3 and that lets you query the ontologies with mappings you create using Protege and -ontopPro- directly from Java.

Some of the features available in QuestOWL (in addition to the general Quest features) are:

  • A simple JDBC-like interface for connections and queries 
  • OWLAPI compatible resultsets 
  • Possibility to generate N-triple files and OWL ABox assertions out of mappings

The file is available in the usual location.

Documentation

We also prepared several examples on how to configure and use Eclipse to program with QuestOWL. Check them out and and let us know what you think!

Cheers,
The -ontop- Team


Tuesday, August 21, 2012

Exciting news

There are a couple of things that will come in the next release(s) of Quest that have us so excited that I had to share them.


  • New query rewriting algorithm (w.r.t. TBox). Yes, finally we are updating the query rewriting algorithm with a totally different one that goes perfect with the rest of the optimisations we have (both in classic ABox mode and virtual ABox mode. As you have seen already, Quest is generally able to produce VERY small SQL queries, often one single SQL query thanks to the way it optimises mappings and how it stores ABox assertions (semantic indexes). However, in some cases generation of SQL takes some couple of seconds, in heavy cases up to a minute. This is due to a very outdated TBox query rewriter. With the new rewriting algorithm there won't be anymore waiting. The basic implementation is already done and the results are amazing, the same simple SQL query, but now in milliseconds/nanoseconds instead of seconds/minutes. With a bit more testing we will incorporate this in the next build.
  • New mapping assistant and mapping extractor. Yes! finally some help in the mapping process, which right now is very painfull. The new automated extractor will be conformant with the Direct Mapping specification from the RDB2RDF recommendation (this one is half way done). The mapping assistant (in development by Josef) will be a manual, but will speed up mapping development by reducing it to a bunch of points and clicks. Of course, free mappings will still be available as before.


Stay tuned! Nice things are coming!



We are pleased to announce the availability of new material. Today we have made available:

  • A demonstration video [1] that shows the basics of -ontopPro- and Quest in 'virtual RDF' or 'virtual ABox' mode. We show how ontologies with mappings work as well as the performance you can get using Quest over a large Postgres database.
  • A Protege 4.2 bundle that includes -ontopPro- [2]. This will save you some time when installing -ontopPro-, only unzip and its ready.
  • H2 stand-alone server with all tutorial databases [2]. H2 is a super-light Java-only database that can be run from a single file. We made a zip with H2 and that includes all the databases you need for the tutorials. You won't need to install Postgres or MySQL to experiment with -ontopPro- or Quest.

We are also restructuring and updating our documentation, we will announce new material as soon as it is available.

Cheers,
The -ontop- Team    

p.s. Please let us know if you have any questions regarding the new material or if you would like to see material on some particular topic. We will try to fix/update/add based on your requests.

[1] http://www.youtube.com/watch?v=KHtlARfex4c
[2] Available at our usual download site

Thursday, August 16, 2012

New build is now available



We are pleased to announce that v1.7-alpha build 1692 of -ontop- is now available for download.

The highlights of this build are:

  • ontopPro now supports for Protege 4.2 
  • Quest now supports LIMIT, OFFSET and ORDER BY in SPARQL queries
  • Better SQL optimization in case of redundant JOINs and queries (less redundancy and simpler queries) which yields improved query execution performance.          
  • Improved support for SQL parsing


Moreover, the protege plugin was extended with many GUI improvements, e.g., better GUI navigation, shortcuts, etc. And last, we worked a lot on improving error management and error reporting in this build. We are sure you will find this build even more stable than before. The full changelog can be found here [1].

Demo

An online demo video of -ontopPro- and Quest in virtual ABox mode can be found at [2].

Performance and Benchmarks

We would like share some exciting news. Our colleagues in Manchester benchmarked Quest in 'virtual ABox mode' (with MySQL as backend) against Virtuoso and MySQL in a very large data set. The results are currently under revision for publication and are very encouraging. They confirmed that in many cases, Quest is able to provide much better performance than Virtuoso, usually 6-10 times faster than Virtuoso, and up to 500 times faster in extreme cases. They also found that our performance is almost as good as using MySQL directly, which means that both, the SQL queries we generate are in many cases already optimal. We will put a link to their paper in our website as soon as the publication is officially out.

Next steps

The next release of the system will include two major new features. First we will finally allow for OPTIONAL in SPARQL queries, taking us closer to a full SPARQL 1.0 implementation (only unbound predicates will be missing, and will follow shortly). Second, we will include new 'Mapping Assistant' and 'Automated Mapping' features in -ontopPro- that will help you speed up the mapping development process.


Best regards,
The -ontop- Team

[1] https://babbage.inf.unibz.it/trac/obdapublic/wiki/ObdalibPluginChangeLog 
[2] http://obda.inf.unibz.it/protege-plugin/




                                                 

Wednesday, August 8, 2012

Demonstration Video

We just uploaded a demo video about Quest in virtual ABox mode. The video explains some of the advantages of doing SPARQL query answering with Quest in virtual mode and it also explains the meaning of the mappings. We use -ontopPro- to show show to create mappings in a scenario based on the SQL version of IMDB.


The embedded version (its better to watch in fullscreen):




Tuesday, July 31, 2012

Delay longer than expected

We are a bit behind the schedule, but we will release the new build shortly. Sorry for the inconvenience ;)

Friday, July 13, 2012

Release delayed one week

We will have a small delay on the release that's scheduled for this week. We will postpone it until next week to be able to clean some last minute issues. Stay tuned! this release is packed with goodies, including Protege 4.2 support, support for LIMIT, OFFSET, ORDER BY and many other things!

Wednesday, June 13, 2012

v1.7-alpha-b1529 is now available

A new build for v1.7-alpha is now available. This time we focused on improving user interaction and solving important bugs/issues related to stability and we are happy to say that this is our most stable and user-friendly build to the date.

Highlights of this new build include:

  • Improved error reporting and error message in Quest
  • Improved syntax error reporting and messages in the mapping editor of -ontopPro-
  • Fixed critical stability issue that triggered an endless loop of error dialogs in Protege 4.1
  • Better layout, improved fonts and other GUI enhancements.
For a detailed description see our changelog.

As a hint to whats coming on the next release:

  • Quest support for LIMIT, OFFSET and ORDER BY in SPARQL queries.
  • Extending FILTER functions
  • First release of the QuestOWL and -ontop- for OWL, the OWLAPI 3 libraries that will allow you to use Quest and -ontop- directly from Java, using the OWLAPI 3.

Tuesday, May 22, 2012

OWLED and ESWC invitation

Next week, during OWLED, we will have a presentation about Quest. After that I'll stay for ESWC. This would be a great opportunity to meet people interested in OBDA and -ontop-, to have discussion and/or hands-on sessions. Please contact me if you would be interested in something like that, we can arrange some time during the conferences.

- Mariano

v1.7-alpha-b1479 is now available

A new build for -ontopPro- v1.7-alpha is now available. Highlights of this build include:

  • Support for UNION in SPARQL queries.
  • Support for SQL Server (needs testing)
  • Improved T-mappings optimization, new produces much fewer and faster SQL queries in virtual ABox mode.
  • The SQL analyzer now understands IS NULL and IS NOT NULL
  • Fixed support for DB2 and Oracle in virtual ABox mode.
  • Multiple bug fixes 
For a full description of the changes see our changelog.

Thursday, May 17, 2012

New build next week (UNION and COUNT!)

A new build will be coming next week with new whistles and bells:

  • First version with support for UNION and COUNT 
  • Support for SQL Server
  • Better T-mappings algorithm, hence faster and MUCH smaller SQL queries in virtual ABox mode.
  • Better SQL parsing (hence better SQL performance in virtual ABox mode)
  • Fixed support for MySQL, Oracle and DB2.
  • Fixed uppercase/lowercase issue with DB schemas.
Stay tuned!

Friday, May 4, 2012

Build 1446 now available

Build 1446 for v1.7-alpha is now available. This build fixes a small bug where mappings that introduce concrete individuals would not be rendered properly by -ontopPro-. The kind of problematic mapping would be:

<"&:;people/{$id}"> :knows <http://physics/Stephen_Hawking>
SELECT id FROM physicists

The problem would rise when there is no prefix for any fragment of <http://example/physics/Stephen_Hawking>

We also added coloring for individuals in mappings.

New paper available

A new paper describing Quest (our reasoner for OWL2QL/RDFS and SPARQL) is now available here. The paper describes OBDA in general, and the functionality and optimizations available in Quest.

Thursday, May 3, 2012

-ontopPro- v1.7-alpha build 1440 is now available


Dear -ontop- users,

We are pleased to announce that v1.7-alpha build 1440 is now available for download from our website. This build fixes several bugs from the previous build including a severe bug that crashed the plugin in some system configurations.

It is highly recommended that everybody upgrades to the new version.

We would also like to take this opportunity to invite you to interact with us. We are working hard to stabilize the current alpha version, and to finish all features required for you to use -ontop- in your applications. We would appreciate it if you could send us issues, bug reports, suggestions or general feedback you may have.

Even better, we would love to have live discussions and/or hands-on sessions with anybody interested. Please contact us if such an event would be interesting/useful for you.

Best regards,
The -ontop- Team

Tuesday, May 1, 2012

-ontop- of KR and OWLED (ESWC)

In the next months we will be presenting Quest, -ontop- and research related to OBDA at the KR conference in Rome and at the OWLED workshop in Greece (collocated with ESWC).

We would be really excited to meet people interested in -ontop- and OBDA projects in general. Please, if you would like to arrange a gathering contact us. We are open for anything, e.g., tutorials, hands-on sessions or just a light chat ;-)

See you in Rome and/or Heraklion!

New build for v1.7-alpha this week

Quick update, this week we will have a new build available that fixes a serious packing issue we had in the previous build. If you where using WindowsXP or Linux and couldn't start the plugin due to NULL pointer exceptions the new build will solve the issue. 

Stay tuned ;)

Tuesday, April 17, 2012

Binaries for protege plugin available, documentation on the way

Quick update, the binaries for the latest build of -ontopPro- (the new name for the OBDA plugin for Protege) are now available for download! We are now updating all the websites and documentation to describe all the new features. As soon as we finish updating the documents we will make the official announcements.

Friday, April 13, 2012

Version 1.7-alpha is almost out!

We are very very excited to announce that we are about to release a new version of the framework. Stay tunned in the next days for a totally redesigned OBDA plugin for Protege, super fast and feature reach Quest and much more!

Saturday, January 7, 2012

Quest Performance for 1.7

Just a small teaser on the performance improvements that you can expect for the next release. We are so excited about them that we can't wait to tell.

Query containment: A critical part of query optimization in Quest relies on using query containment (CQC) to remove redundant queries from query rewriting. This is done by mutual checks, query1 vs query2 and so on. In complex cases, Quest might need to perform thousands or hundreds of thousands checks for a given query, hence the performance of the containment check algorithm is CRITICAL. We have been improving our algorithms and API to minimize the cost of this, so far we have a achieved a reduction of up to ... 95% of the cost! This means that if you got a complex query that was taking 12 s or 20 s to rewrite due to CQC, now it will just take 0.5 s or 1 s!

The best part is that what we have implemented until now is just a small fraction of all the optimization related to query containment we have in mind!

SQL Analysis: Up to version 1.6, we handled SQL as a black box. Any SQL query that was present in the mappings, was not really understood by Quest. The only thing Quest got from the query was the "signature", that is, the columns in the SELECT. Because of this, during the generation of the final SQL queries, Quest always had to rely on nesting of sub-queries. For example:


SELECT view1.x view2.y FROM 
 (SELECT x FROM employee) view1,
 (SELECT x,y FROM worksfor) view2
WHERE 
 view1.x = view2.y

This is, in general, not good for performance. In order to plan this kind of query, the DBMS like Postgres, DB2 and Oracle have to "flat" the query. If this is not done, chance are that the query plan for the query will be not good, index might not be used, join orders might be suboptimal. What is worse, if the DBMS doesn't implement query flattening, for example MySQL, then the nested views have to be materialized before usage, with no indexes or anything... VERY BAD PERFORMANCE.

The good news is, for version 1.7, Quest will include its brand new SQL analyzer, and updated code that makes use of the result of this analysis during query rewriting, and SQL query generation. The first version of the analyzer doesn't understand 100% of SQL, however, it is enough to cover around 80% of the SQL queries we have seen in most use cases. The result is that the SQL queries generated by quest will be much closer to what a human would write on its own, and the DBMS will be able to optimize much much better.


If you would like to give version 1.7 a test drive before it is released, please contact us.