Gabriel Morell


A hole, for things

ThingHole Intro
posted June 23, 2017, 2:17 a.m.
under thinghole · perma

ThingHole is the center of the ThingCosm and the first tool that was written. It was written with multiple instances of any service running in parallel from the start., thinghole has its base services written in twisted with a Django front-end for management, configuration, and data visualization.

The twisted services are:

  • Horizon which ingests data,
  • Stream which deals with passing the data along to the front-end
  • Manifold which specifies triggers to pass the data onto other services.


The horizon

The horizon is a service that collects arbitrary data and inserts it into the database as well as passing it along to the other services such that they don't need to access the database. For speed reasons, the horizon service connects to postgres and loads all the auth tokens into memory and continues to update the in-memory cache every thirty seconds. On data received the horizon inserts the data into postgres with [code] as well as passes the data along to the various stream and manifold workers.

The stream

The stream service receives data from the horizon services and forwards them to connected web socket clients for charting and tabulation. It makes sure that it passes on the values to the correct connected client. The data is also converted into the various formats for each charting and tabulation feature.

The manifold

The manifold service passes on data to other services based on user described arbitrary rules, either to a crossbar topic, another thinghole reciever, or arbitrary URLs with programatic injection of additional data and headers