For both better and worse, TradeLens has been among the very first adopters of APIs for the shipping industry. It’s meant that we’ve been though our share of trial and error while iterating our designs. But, now, a couple of years in, our APIs have largely stabilized. So it’s a good time to reflect on and share some of what we’ve learned.
Architectural balance
One of the most significant architectural features we’ve had to test is the balance of RESTful versus event-oriented APIs. These are hot paradigms these days, and we’d like to share how we found they apply best to the shipping industry. To be clear, TradeLens uses both paradigms—this is in no way about picking sides, but just choosing the right approach for different use cases.
Among our obvious REST use cases are subscription APIs. You create subscriptions (POST), query your subscriptions (GET), update them if your webhook end point changes (PUT), and remove them when no longer needed (DELETE). That’s just straight-up REST.
The eventing paradigm
When we get to the use-case in which one of the platform contributors needs to publish a shipping milestone, it’s a different story. Here, we have the common industry transport milestones: containers gate-in to terminals, get loaded on vessels, depart from terminals, etc. These exist throughout the industry already, and have been modelled as “events” in IT systems literally for decades.
Milestones come from a diverse array of parties to shipments, like truckers, terminals, and vessels, that do their job and simply announce the that they have completed a particular task, like “I just gated in a container.” They do this without necessarily knowing other details, like what’s in the container or what its final destination might be. This heterogeneous fire-and-forget situation fits poorly with REST, which firstly assumes that you know the existing state which you manipulate and then have an opinion about how the provided data is to be kept and changed based on create, read, update and delete CRUD operations.
Enter the event-driven paradigm, which allows every participant to publish their view of milestones as they occur. No assumption is made about how this information is to be used — it’s the receiver’s responsibility to apply proper interpretation, business rules and even data modelling. Events can even contradict each other (for example, they could disagree about when a container arrived), and again it is the receiver’s choice to decide what to make of such contradictions.
Context spans across each paradigm
Although various actors can publish their own isolated events, that doesn’t mean there isn’t any kind of context. Some of the parties to a shipment are in fact responsible for the bigger picture. In this case, a consignment has been booked and its transport equipment has been allocated (“consignment” and “transport equipment” is UN/CEFACT terminology for shipment and container. I’ll cover this language in more detail a separate post). Then the container starts to move and generate milestone events.
The consignment and the transport equipment are thus contextual entities and therefore excellent candidates for REST. However, again it depends on the use case. But we can’t just suddenly introduce a REST-ish representation of, say /Consignment (we tried!). The reason is technical: events are implicitly asynchronous. There’s no guarantee of when actors send in their pieces of the puzzle. And also, throughout event architectures you find technical messaging queues, which only guarantees they’ll be consistent eventually. Attempting to perform RESTful state manipulation on an asynchronous system results in updating an outdated state due to race-condition situations.
So, on the eventing side, notifications to actors that consignments have been booked are published asynchronously, just like any other (milestone) event. And again, receivers implement how to react on that based on their own business logic, processes and data.
Context-free publishing is much simpler for carriers
But there’s another argument for event publishing beyond the somewhat theoretical (but very real) race-condition concerns. Simplicity is key for the shipping industry and a guiding principle in our assessment of API paradigms. With REST, in order to do a PUT or PATCH, you need to know the details of the state change. That’s theoretically possible, but practically quite cumbersome, and events provide a simpler approach. Within standard ocean carrier systems, data flow from an operational system (which governs state) through layers of eventing platforms. We have only ever seen the new state expressed in these internal events, so the only practical way to calculate changed state (diff operation), is to keep a full copy of the model at the API interface code. In developing our APIs, we found that there’s no need for all senders to keep such state when TradeLens already has it and can do the diff operation on behalf of everyone, greatly simplifying the interface-code-to-basic-data transformation.
Back to REST
That said, as hinted earlier there is a case for supporting RESTful actions like GET /Consignment. And indeed, TradeLens provides it, too. It returns the consignment’s transport equipment, planned, estimated and actual events. The important difference is that this is a request-reply use case: what we return reflects what we know at the point in time when you asked.
In order to distinguish clearly between the mentioned use cases, TradeLens exposes them under individual APIs, respectively:
When it comes to RESTful and event-driven APIs, each plays their part. These have been some of our hard-earned but extremely interesting lessons from entering the API-era in the shipping industry.
Learn more on the TradeLens API Documentation page. We hope that these learnings can be of use and inspiration to others — and we welcome all thoughts and comments on our approach.
.svg)
