The Underlay is a project to make machine-readable public knowledge available to all as a public good. Learn more.
Knowledge is often exchanged in formats optimized for computers, and then used to render webpages, maps, diagrams, tables and text for human consumption. It is also used directly by machines to navigate vehicles, trade stocks, control appliances, design structures, formulate scientific hypotheses, order search results, and much more. Because this machine-readable data is not in a specific human language, it easier to share it globally. But today, most of these machine-readable data sources are privately held and controlled, and the ones that do exist publicly are fragmented and don’t work well with each other. The goal of the Underlay is to improve the way that machine-readable knowledge is shared and used for the benefit of the entire world.
The Underlay represents knowledge in the form of relationships between entities, for example the relationship between an author and a book, or a location and its current temperature. This method of representing knowledge is call a knowledge graph, and it can be used to represent any type of machine-readable information. The Underlay differs from most other knowledge graphs in that it represents not only the information, but the provenance of that information: where came from and what others have to say about it.
The Underlay is not a single piece of software, but rather a suite of interfaces that allows anyone to publish data for others to use, and to connect data to other databases, which can then be queried and used in applications. Users of the Underlay can pull specific information from the Underlay into databases that contain only the information that they judge to be trusted and relevant for their specific needs. Thus, the Underlay does not replace other databases, it makes it easier to build databases and keep them up to date.
The Underlay draws its content from public information sources, including public records, scientific publications, commercial catalogs, the web, and open databases created for the public good. The Underlay also provides a convenient way for for-profit, not-for-profit, and government organizations to publish machine-readable information that they wish to make easily available, including schedules, catalogs, scientific datasets, public disclosures, standards, and more.
The Underlay works by allowing anyone to make assertions about the world, including false ones. Critically, the Underlay tracks the source, or provenance, of these assertions. Unlike the explicit existence of a data file, knowledge is nuanced, dynamic, and open to qualitative judgment. As such, the Underlay is designed as much around keeping track of who asserted a particular bit of knowledge as it is the knowledge itself.
Knowledge in the Underlay is not assumed to be true because it has been asserted, but rather, is designed to be curated by an external group of humans or algorithms that organize assertions into a trustworthy set (for any relevant definition of trustworthy). When querying data or building applications, users of the Underlay are able to see exactly who asserted the information they’re relying on, and how many other people agree with those assertions, allowing them to make informed judgements about the quality and veracity of the data they encounter and use.
In addition to querying the data in the Underlay and using it to build your own databases, the Underlay is designed to power applications, called overlays. The Underlay’s focus on data interoperability simplifies development and preserves the context behind every piece of data, automatically giving users the ability to see what else has been said about it. Example overlays might include:
A newsreader that suggests rhetorical tropes and sources for casually-sourced claims, and allows article authors to confirm those suggestions.
A browser for the collection of scholarly articles and patents, with their joint citation and co-author graphs, and extracted topics.
A generative adversarial network (GAN) trained on images and models of every item excavated from an archaeological dig in a region, along with the algorithms used.
The Underlay is built and maintained by an open Consortium. Besides gathering data and creating tools to import it, the Underlay Consortium is also developing software to enable Consortium members to choose which parts of the Underlay they will store and which new assertions they will accept. These servers will keep one another automatically updated. They will also allow Consortium members to directly the update their own databases based on their own criteria of trust, utility and relevance. They will be free to use and republish the data in any way they want, including making it available to non-members. We expect some Consortium members to be for-profit-corporations, and others to be not-for-profit member that serve either a specific constituency or the public at large. Non-members will access the Underlay indirectly, through Consortium members.
To inquire about joining the Consortium, send a note to firstname.lastname@example.org.
We are currently developing the first specifications and implementations for the Underlay. To stay in the loop, subscribe to our very infrequent newsletter.
Yes. We are currently early in the development phase, but could use help from developers who want to provide feedback on our initial specifications and code. To get involved, send an email to email@example.com.