Why your production REST APIs are incoherent and messy?
Teams set out with the best API specs in design documents and a year later the APIs look inconsistent and messy. This situation gets amplified as APIs across all your microservices start showing a similar pattern.
The reason is not you. REST is fundamentally flawed — here is why.
Story line
REST fails to standardize the creative space
Standardization bodies like the IETF, W3C establish standards and specs for fundamental systems like HTML, HTTP, and others.
HTML or HTTP is one tiny piece of the puzzle. putting up a cat video on the internet takes a surprising number of layered systems. Along with the layers of the technology stack, there are numerous standardizations.
The lower technologies in the pyramid above provide relatively less creative space. Most engineers do not delve to that level during a normal day or product development.
However, the highest layers are inherently creative. It is really hard to set an exact spec/standard on a website layout. One may prescribe the accepted colors and tone of the copy. But the final outcome from the two developers will be different.
Enforcing standards are hard where there is room for a lot of creativity.
The same is true for REST APIs. There is a lot of creative space on top of the URIs or headers to build robust APIs solely using REST guidances.
The promise of REST
REST was introduced In 2000 by Roy fielding as part of a PhD thesis. API was the contract between the backend and the frontend systems to “transfer state”. HTTP was already providing the core component of data transfer protocol between server and client.
Representational State Transfer (REST) was an architectural proposal of how to use the 3 HTTP components (URI, header, body) to create uniformity when building applications that represent server state on the client.
This proposal may have been a fine proposal then, but the internet has outgrown these recommendations.
There are five primary goals of the REST system — Client-Server isolation, layered system, stateless, cacheable, and uniform interface.
Client-Server isolation — allows us to swap either of them. We see android, IOS devices, IOTs, laptops, and desktops driven by mostly similar APIs. However, this comes out of the box from HTTP 1.1 and is not specifically an addition with REST.
Layered system — is also an architectural pattern for the client to be unaware of the layers serving the data. Not a particular innovation of REST and could be achieved at the level of TCP.
Stateless — Each request from the client to server must contain all of the information necessary to understand the request — The areas where it deviates are in cases where we keep server session data for security and authentication
Cacheable — Requests should allow caching. There seems to be a fair amount of success achieved in this area.
Uniform interface — We failed almost every aspect of the uniform interface. Resource identification, Resource manipulation, Self-descriptive messages (especially on errors) are things that are least standardized when developing using REST.
Shortcomings of REST APIs
REST, perhaps did not aspire to be an all-encompassing standard or guidance for building the business application. We have just embraced it blindly to be just that.
Here is a list of common problems with REST APIs
Large get URL - Sometimes a get URL has too many parameters. We see developers convert these semantically get calls into a POST request. We can see this done in various applications with are heavy with search filters.
GET side effects - In practical applications we see GET APIs emitting events (Kafka or any other system). If these events were used to calculate anything of business value, GET is not without side effects.
Complex Objects in URL - a GET request does not have the ability to send arrays, hash, or sent via query URI. Different languages & frameworks interpret it differently. The REST principles of uniformity or client-server interoperability are broken in these cases.
Explosion in resource Identifiers - Complex business cases have many ways to access a resource. E.g., firstHotel, lastHotel, firstHotelVieableByCustomer, firstHotelForKayakIntegration, firstHotelForPricelineIntegration. While it is idealistic to want to put it into /hotel?[parameters]. I bet you to convince your engineer to do that. Testing, blast radius isolation, legacy code, or just completely different semantics might push us to have multiple URIs for the same resource.
Bidirectional data - With the rich nature of apps today, the server constantly sends data to the clients over persistent connections. There is no REST guidance to rest upon.
Business Errors - A user’s credit card payment API errors due to credit card failure, should the API return a 200 or 400 error? you might think 400 as the card is wrong. What if it was due to previous arrears and not due to any input in the current request, would it still be a 400 even though it was not a client error w.r.t this request?. As you can see, issues like this are very common in practical applications with no clear guidelines.
HATEOAS - The concept of recursive explication might work for some basic graph data models but it is not practical for a majority of business applications. it further creates chattiness between the server and the client. This is the worst form of N+1 query between a highly latency-prone network layer. Most practical applications stay away from this ‘REST’ feature.
APIs per consumer - As the number of devices and experiences increase, the flow between those devices starts diverting. E.g., your experience of booking an airline from the app might be different from that on a desktop. As the experiences change, the APIs will need to be different. With this, the reusability is out of the window and the need for multiple APIs looking similar happens.
Versioning - No one has come to terms with one approach that is easy and has meaning. Unlike server for software, versioning is not as easy in APIs as it spans multiple dimensions like schema, logic.
We have relied on REST for almost 20 years and it is time for a change. We have to consider the complexities we see in our business today and ideate a new standard from the ground up. This time, we should put practice into theory and not the other way around.