OGC’s Testbed 13 Demonstration

Last week, almost 100 attendees gathered at USGS’ headquarters in Reston, Virginia to attend the demonstration of the outcomes from OGC’s latest annual ‘Testbed’ initiative, Testbed 13.

T13 Demonstration at USGS 2017

Like at other OGC Testbed Demonstrations, the demo was centered around a ‘scenario’ where the new technologies could be shown in a practical context. This year, the scenario was ‘mass migration’ and how geospatial technology can aid the organisations looking to help displaced persons return home after conflict has ended. In addition to this scenario-based demonstration were also a number of technical presentations and open-floor discussions.

But before I get to all of that, I’ll first tell you a bit about just what OGC’s testbeds are.

In OGC’s annual testbeds, major sponsoring organisations put forth geospatial IT problems. OGC staff integrate these requirements into a set of software architectures, and technology providers then receive support and funding to collaboratively research and rapidly develop prototype solutions. Eventually, the results are handed over to the OGC Standards Program where they take the form of consensus-based open standards and best practices.

OGC Testbeds are a key activity of the OGC Innovation Program, and the main means by which the OGC membership, OGC’s testbed sponsors, and OGC’s partner standards organisations have made spatial information an integral part of the world’s information infrastructure.

Testbed 13 took the requirements of 12 sponsors, and, over the course of 9 months, 31 different participant organisations created solutions to 85 work items - which formed the core content of the demonstration event held last week.

The event occured over two days, with the first day providing an overview of the testbed’s architectural and software challenges and newly developed solutions, and the second day involving technical discussions.

On the morning of the first day, a high-level presentation was given by OGC’s Testbed-13 chief architect, Dr. Ingo Simonis, which summarised the 15 major outcomes from Testbed 13 (see below image) and placed them within the context of the aforementioned ‘mass migration’ scenario. After lunch, testbed participants presented their own work across each of the threads.

The 15 outcomes of OGC Testbed 13

The second day of the demo examined the technical achievements of the Testbed, and opened up the floor to discuss the new technologies, potential future directions, connections to OGC Standards Program activities, and the follow up activities required to ensure maximum incorporation of testbed results into the wider geospatial standardisation domain.

With 15 newly developed technical outcomes that encapsulate the 85 work items, it would be very easy to end up with a blog post the size of an Engineering Report. So, to save you wading through pages and pages of text on your holiday break, I shall only discuss the four that Ingo felt were the were the most revolutionary.

Natural language queries and fit for purpose data (expert/non-expert)

“What is the safest route?”

It’s a simple question, and one commonly asked by displaced persons fleeing horror, but how can the expert users in the geospatial community give an answer to those non-experts on the ground? This particular Testbed 13 outcome has the ultimate goal of being able to take ‘natural language’ queries from non-expert users and give them the answer that they’re looking for.

In the context of the Testbed 13 demonstration, the non-expert user wanted to know what would be the safest route for displaced persons to take to Daraa, Syria, from the Zaatari refugee camp in Jordan, after a ceasefire had been declared.

In Testbed 13, this required some forethought from the geospatial experts, as the data and processes used to answer such a question needed to be pre-defined (specifically, placed in an OWS Context Document) and registered in a catalogue, which the client engine could then match to the question asked.

So at the moment, the geospatial expert needs to be able to foresee the questions that are most likely to be asked. However, this isn’t quite as big an ask as you might assume: more often than not, the data providers have a good idea of what their data can/will be used for, so the geospatial expert could use this insight to package up and register OWS Context Documents for use in just such a search query.

Currently, questions like the above find the pre-defined OWS Context Docs using simple keyword/pattern matching, but the eventual goal is to be able to take a natural language query and bring in the required geospatial contexts on-demand.

However, not all the data required to answer a such question is ready-and-available via a web service (eg an OGC Web Feature, Coverage, or Map Service) - quite often, several processing and analytical functions need to be performed on existing data.

As such, we need to find a method to bring in disparate Web Processing Services (WPS), and chain them together in a distributable workflow - using a method that honours access permissions, yet still appears seamless to the non-expert end user - so that we can create the data required to answer the question posed. Which brings us to the next outcome of Testbed 13:

Not all data exists behind WFS/WCS: secure workflows and dynamic WPS chaining

Here it was demonstrated how a geospatial expert can create a common workflow involving numerous WPS processes from different providers, and additionally share that workflow with others who may not have the expertise or resources to use those processes individually.

Using Business Process Model Notation (BPMN), it is now possible to chain together the separate WPS to create a spatial data workflow, and in turn make that workflow available behind another single WPS, which can then be registered (and discovered) in a catalogue.

Of course, each of these WPS belong to different security environments, so Testbed 13 was able to demonstrate a method (using OAuth and OpenID Connect) to mediate a user’s identity and handle all the security aspects on their behalf - making a secure interaction possible without requiring frequent, separate logins. This helps with the ‘shareability’ of the workflow, too, as nobody without the appropriate authority will be able to access any WPS that they shouldn’t be able to, even if they ‘install’ a workflow that uses them.

Indeed, the packaging and distribution of ‘workflows’ or other geospatial applications is beneficial to experts and non-experts alike - especially if they can be used in a way that takes advantage of the scalability of the cloud. Which brings us to the third outcome of Testbed 13 that I’ll be covering in this post:

Applications in the cloud

A recurrent theme of the Testbed 13 demonstration was a way to empower the non-expert user to find answers to their queries. One way that this can be done is in the creation of a sort of ‘App Store’ for geoprocessing and other geospatial applications.

Indeed, Testbed 13 demonstrated a way that an expert user can create a geospatial processing application that can be packaged into a container (Docker, in this case) and registered on a hub. Expert and non-expert users alike can then browse the hub for applications suiting their requirements, and then deploy them onto any cloud infrastructure and at any scale.

This involves two components, both of which are WPSs: one is the ‘App Store’, where you register your new app and users can discover it, and then there is the ‘Cloud Engine’ that does the actual deployment and execution of the app in the cloud. The ‘Cloud Engine’ has to be created specifically for the cloud infrastructure that it will be deployed on, such as AWS, Azure, Google Cloud, etc, because each cloud has its own functionality and technology behind it. However, once the engine has been created, it’s just as distributable as the ‘app’ itself.

The final outcome of Testbed 13 that I’ll cover revolves around a similar concept of packaging up geospatial information workflows for use by non-experts - specifically, to make it possible to build easy-to-configure ‘what-if’ parametric models for use by non-experts - even if the models require interaction with other complex ensembles of data models, such as those concerning forecasts of climate change.

Looking into the future: Geo-simulation

The idea behind this outcome is that there are a multitude of users that could benefit from the data generated by ensembles of complex models - such as climate models - but don’t have the scientific training to know how to interact with them directly, nor the need to access them at the scales that they cover. The example used during the demo was of a farmer that wants to know how his or her crop yield will change over the next decade, given different climate change scenarios.

As any climate scientist will tell you, understanding how the climate will change over the course of time involves using an ensemble of different models to produce a number of different outputs, which can then be used to gain an understanding of the more ‘likely’ scenarios. However, a non-scientist doesn’t have the technical training, nor the desire, to interact with ensembles of forecasting models like this. This user needs to access only a subset of the climate data relevant to their farm, and ingest it into their local GIS to be overlayed with their own data.

As such, this outcome was focussed on using Web Coverage Services (WCS) for efficient subsetting (both spatially and temporally) of large datasets - specifically the optimisation of the provisioning of climate change data.

This subsetting of data occurs in such a way that the non-scientist can do simple parameterisation (such as different possible rainfall scenarios) even though in the background such a query may require outputs of much more complex models (climate change, seasonal variation, etc) in order to generate the desired result.

This efficient subsetting will actually make life easier for experts and non-experts alike, as it can be used to streamline the process of making available the immense amounts of data required for input into the types of models used by experts, too.

Making complex climate data available to non-climate scientists has huge potential impact because it allows geospatial users like political scientists or human geographers to ask questions such as “what will happen to the 6 million people in this area when the climate starts to turn their agricultural lands to desert?” without having to deal with the terabytes of NetCDF or WCS data that climate models produce and rely on.

With even more new technology demonstrated and discussed than I could cover here, the Testbed 13 Demonstration event was clearly a great place to discover new geospatial technologies that improve the sharing and integration of spatial information, which in turn has widespread and longstanding benefit for testbed sponsors, geospatial users, and society at large.

If you’re keen to learn more about the outcomes of Testbed 13, the Testbed 13 page on OGC’s website currently includes links to powerpoint presentations from day 1 (including Ingo’s overview), and will soon be updated to the video recordings of said presentations, as well as a detailed summary report: www.opengeospatial.org/projects/initiatives/testbed13.

If you’re of a heavy technical bent, keep your eye out for announcements from OGC regarding the availability of the Testbed 13 Engineering Reports, which are being reviewed and finalised as I type this.

And finally, if you were unlucky enough to have missed the demo event in Reston, a second shorter presentation will occur at the ESIP Winter Meeting on the afternoon of January 9, 2018 in Bethesda, MD. Registration is available at the link above.

Dr Luis Bermudez leading a discussion on day 2 of the Testbed 13 Demonstration

Dr Luis Bermudez leading a discussion on day 2 of the Testbed 13 Demonstration event.