twitter feed

Each day you get thousands of content items from the feeds you subscribe to

You go over them one by one, waiting for the ones you want to read.

baggage carousel

That's just fine if you have all the time in the world

Or just looking for some entertainment, totally not caring that some hidden item in your feeds contains information that could change your life.

supermarket

What if you could browse through your feeds just as you shop in your supermarket?

After a few times, your mind will learn how things are organized, so you’ll be able to find & discover the information you want in seconds, without missing a thing.

Hey, welcome to our project

  • Prior to the Web, information & data was available in files & databases, located on specific computers & networks, but it was extremely difficult to access it
  • The Web introduced an access layer - a uniform standard way to access any information & data from files & databases around the world
  • The problem is that although we now have access to tons of valuable information & data, we only have time for consuming a tiny fraction of the data we subscribe to, & therefore only making a tiny fraction of its potential value
  • The reason for this is simple: the Web may have solved the problem of accessing data & information, but it didn't provide any help with processing & aggregating the information & data, or an interface to consume it effectively
  • The Web Wide Matrix is an abstraction layer over the Web, that tries to solve this problem using:
    1. A new uniform way to access information & data
    2. A mechanism for easily processing & aggregating data
    3. An interface for effectively consuming information & data

1. A new uniform way to access information & data

The problem

  • There are millions of data-islands across the Web, & each one of them has its own way of accessing the data & making sense of it.
  • This means that you need extensive manual work to be able to connect to, understand, & normalize each data source, & only then can you start processing, aggregating & merging it with other data sources.
  • Furthermore, the process of discovering the available data islands & comparing them is also a laborious manual work.
  • There were many efforts to add semantic layers over the Web & link data across islands, but so far none of them has proved effective in solving this problem.

Solution overview

  • The Web Wide Matrix offers a completely new access model to the information & data across the web, allowing simple uniform access to any data island
  • This access model also takes into account that the data continuously evolves & updates, so the way you consume it isn't by taking a snapshot or run some query at one point in time, but rather by subscribing to the stream of updates
  • Another powerful feature of the model is that you get an overview of the available data sources & can easily compare them & discover the data that you want

The model

  • In the real-world, you have maps for organizing & discovering the available resources. If you need to buy some product, you look up the address of a store that sells it, & then go to that address to buy the product
  • Similarly, within a place you have maps that help you find the exact item that you want
  • The Web Wide Matrix is an abstraction layer over the Web, based on a spatial model similar to the real-world
  • It is based on the concept of Composite Buildings, for storing data
  • It starts with the ground level, on which you can have buildings, whose address is their coordinates
  • Each building contain metadata & a payload of data
    • If it's just one piece of data, e.g., a JSON document, the building is called "atomic building"
  • The building can also have floors, which also contain data - for example if there's a newer version of the data, it will be available in a new floor
  • But buildings can also have nested buildings inside them, whose address is based on their coordinates on the floor in which they are located
    • Such buildings are called: "composite buildings"
  • In addition, floors within buildings can be divided into areas, called Rooms, for grouping related data together
  • The way buildings are connected to the Web is using special "data-pipes" that continuously stream data from some data-source on the Web into a building
  • The content of buildings is following a common machine understandable format, so you can access & process all data in a uniform way
Buildings
Composite building
Building servers

2. A mechanism for easily processing & aggregating data

The problem

  • A lot of data is available on the Web, but in order to process & aggregate any piece of data, you need to manually develop the software for processing this specific data
  • This is similar to how before the Web, you had to develop the code & infrastructure for connecting any 2 computers for allowing you to fetch some data from one computer to another
  • The Web standardized this access model & made it easy to access any data around the world, but it didn't standardize the way to process this data

Solution overview

  • The Web Wide Matrix doesn't treat data as just static information that is stored & exposed, but rather as something that needs to be continuously processed & manipulated to extract value from
  • This is why it doesn't just model the data, but also the processing layer as part of the model
  • The processing is done by software agents that live inside the Matrix buildings, & are hence called Residents, whose purpose is to process & organize the data - users tell them what they want, how they want the data to be organized & what value they want from it, & they continuously process the data to get the users what they want

The model

  • The way the processing model is combined with the access model is using a layer inside the buildings that captures the processing state & value potential of buildings. We call it the "building smell", & its made available & propagated around buildings
  • Residents move from building to building, attracted by their smell, & apply actions to process the data inside the buildings
  • The seed knowledge of how to do this processing is provided using something called "Training courses", which are very easy to develop, & also available for download from vendors
  • But beyond the seed knowledge, the Residents also learn thru feedback & exploration how to improve in processing data to satisfy users
Resident hackers
Processing flow
Hovercraft powering resident hackeres & data-pipes

3. An interface for effectively consuming information & data

The problem

  • The common abstraction of data is that of files & folders, data spreadsheets & letters
  • The inherent problem with this abstraction is that all of these are based on reading pieces of data one by one
  • This can only work if you have a very limited number of items
    • The metaphor we like for it is a baggage carousel in airports, that forces you to look at all items until you find what you want. This is only effective if you have few dozen items & not thousands.
  • The problem is that the amount of data we have today - user generated, device generated &c - is huge, & can't be effectively consumed in the current abstraction & interface

Solution overview

  • The Web Wide Matrix doesn't just address the access & processing of information & data, but also how to effectively consume it
  • In the real world we have 2 cognitive powers that we utilize all the time in order to perceive & find stuff out of thousands of items
  • The 1st is something we call "Top-sight vision", & is the ability to look on a scene that contains thousand of items, & find what we're looking for in a few seconds
    • For example, if you are in a big department store, as long as you can see the different isles, you'll quickly see where the shoes or coats are
  • The 2nd is something we call "Spatial memory", which allows us to remember where we last saw something
    • For example, in your house you remember where you left something, or where the medicine or tools are
  • In order to leverage these 2 cognitive powers to consume large amounts of data & information, we need a different abstraction of data, which is more similar to the real-world.
  • We call such type of interface an "Organized Space", & its basically a VR representation of data, allowing users to see all the data, quickly discover what they want, & also remember where they last saw something to find it or similar items again

The model

  • In order to provide an organized space interface to its data, the rendering aspect of data is built-into the model
  • Each building also have a visual 3D representation chosen to represent the data inside it
  • When you look at the buildings within some floor, you get an overview of the data inside it
  • Part of the work done by the Residents is to organize the data buildings within floors & rooms into an organized space, so that users can easily consume them, just like they easily discover & find stuff in supermarkets or stores, that also contain thousands of items
  • The actual way users consume the data in the Web Wide Matrix is using a software tool called the Matrix Browser, which renders the organized spaces in VR interface, that will be available on many types of devices
Organized Space
Raw feed
Service providers buildings

Learn more on the underlying problem:

[Watch the video]

Learn more on the Web Wide Matrix solution:

[Watch the video]