Rotonda currently has no real transport that’s meant to export whole tables or RIBs. Sure, you can use the HTTP/JSON API or use the MQTT transports. These will never be efficient for large volumes and/or high-bandwidth though, because of the unavoidable amount of serializing (and deserializing on the client side!) that needs to done to satisfy these transports.
Serialization is costly in both space and time. We will have to allocate for units of serialization. Even HTTP chunking (“streaming”) and compression will only get you so far. Compression still takes CPU time, and will not alleviate the serialization, it comes on top of it. The client will have to do the reverse process. Chunking done over multiple threads will reduce time spent, but has issues with, for example, ordering. This is all the more serious, since a Rotonda instance has also a diverse range of other tasks to perform in near real-time, mostly around the ingress of BMP/BGP feeds.
What we would like to have is a transport protocol that doesn’t do any serialization, and is as much zero-copy as possible. The bytes of the data should be stored in memory in a way so that they can be copied out:
- one time, or with the least amount of copies possible (“zero copy”).
- with as little transformations as possible (“no serialization”)
- in as large chunks as possible (“locality of reference”).
This would enable a protocol that can be used in a setup of two Rotondas instances: one for collecting the feeds into memory (and/or disk, that’s another discussion), and then forwarding the data over the wire (or socket) to another Rotonda instance. The receiving Rotonda does not keep (persistent) RIBs or tables, but only does serialization to the desired third-party output protocol/format, like JSON/HTTP, protobuf etc.
We are currently working on this through these work items:
- Reorganize the internal storage (which we renamed to
routedbbtw) to have better locality of reference, and have deduplication of e.g. path attributes. - Reorganize the BGP PDU parser (
routecore) to use zero copy mechanisms as much as possible. - Create the network protocol (
rotoro) to allow for high volume/high bandwitdh transfers between Rotondas.