Hologram's Data Engine is a message queue and protocol translation layer designed to route data from your embedded devices to other internet-connected services. It features a low-bandwidth ingestion protocol designed for resource-constrained microcontrollers. The Hologram Nova's standard library exposes the Data Engine as an Arduino Serial-compatible object for sending data.
A message consists of a data payload and metadata. The payload is typically a text string, but can be arbitrary binary data. Most metadata is automatically added on the server based on the message source, but you can also specify one or more topics to control how the Data Engine routes the message.
Topics provide the Data Engine with context on what the data represents, where the data originated, or why the data was generated. The Data Engine uses topics to conditionally route messages to their proper destinations via routing rules.
Topics are arbitrary strings, and a message can have more than one topic associated with it. You should develop your own conventions on how to use topics. Thoughtful use of topics can make it easier to set up new routing rules and integrations without needing to update device firmware.
User topics cannot begin with an underscore character. The Data Engine reserves these names for system topics such as _DEVICE_ID_.
Here are some examples of good topics to add to messages:
It's not always obvious whether to include some information as a topic or encoded in the message payload itself. The Data Router is only able to route messages based on topics, and messages to downstream services can include topics. Therefore, for the most flexibility, it's a good idea to use topics for classifying messages.
In addition to the topics you specify when writing a message, the Data Router will automatically append certain topics based on the source and protocol of the message.
All system topics begin and end with an underscore character, e.g. _SIMPLESTRING_. For security and consistency reasons, you are not able to specify these system topics explicitly.
The most important system-generated topic is the device topic. Every message written to the Data Engine from a device gets a topic in the form of _DEVICE_1234_, where 1234 is the Hologram device ID.
For a complete listing of system tags that the Data Engine may add to your messages, please refer to the System Topics reference.
When sending a message to Hologram's Data Router via the TCP API, you must authenticate as a specific device. The authentication credentials consist of a 8-byte device key. You may view these credentials under Cloud & messaging on the device dashboard page.
If you believe that a device's credentials may have been compromised, you can regenerate them. Each device has one active set of credentials at a time, so the old credentials will no longer work after regenerating.
The TCP API also supports a separate authentication method based off of the SIM card. This method is used by the Hologram Nova if you use it with the Hologram Python SDK in its default sending mode. With this authentication method, there is no need to generate credentials on the dashboard.