Level-Setting: The Problem
The shipping logistics industry refers to the companies and underlying processes that manage the flow of goods and data from the point of origin to the point of consumption. In 2024, this process typically occurs on a global scale and involves a series of complicated and often manual steps including: ordering, freight forwarding, booking, packaging, pick up, transport, hand-offs, receiving, processing, delivery, and exchanges. The data associated with all of the above must be ingested into, perhaps, among the most complex network of dependencies and relationships there is in the economy.
In considering the global shipping logistics industry, one must take account of trade laws, political boundaries, economic fluctuations, and geographical challenges. They must then combine this knowledge with that of the ever increasing trends fueling complexity such as fragmented channels, increased product variations, consumer demand for customized solutions, and an expanding global footprint among businesses moving more items to more places. With all of these variables and given logistics costs account for 12% of total costs in the US and Europe, 16-18% in India, and 8-10% in China[1], the goal of shipping logistics is to make the complex process run as efficiently as possible.
In other words, the goal of global shipping logistics is to solve what could be considered the ultimate optimization problem.
Enter Machine Learning (ML): The Optimizer
ML enables computers to learn and improve from experience without being explicitly programmed. It involves the use of data and algorithms to imitate the way humans learn, gradually improving accuracy and making predictions or decisions based on the learned patterns and relationships in the data.
Machine learning is great for optimization problems because it is a process of iteratively improving the accuracy of a model, thereby minimizing the degree of error. This enables machine learning models to learn and make predictions about new live data, such as prices, routes, availability, etc, leading to more effective and efficient decision-making in tasks such as resource allocation, scheduling, and routing.
If global shipping logistics is the ultimate optimization problem and machine learning is the ultimate tool for solving optimization problems, it begs the question: why hasn’t adoption been as simple as a key unlocking billions in value?
The Barriers
There are a number of consequential barriers in the way of applying ML to the ultimate optimization problem. Most of these challenges originate from the archaic technologies and processes that characterize the sub-industries playing a role in shipping logistics. These are only amplified by the fact that global shipping logistics is complicated.
In considering the limiting factors to addressing the shipping logistics optimization problem, the problems can be categorized into 1) data 2) regulation and 3) behavior.
Data
Data is the fuel of ML. It is the input into the “optimization machine” and, in the case of shipping logistics, the fuel is clumpy and of heterogeneous types and qualities.
Data Availability: There are significant challenges with the availability of data. In each of the previously referred to stages, there are a number of firms providing services. These firms may be part of the same supply chain but it is often the case that they do not share data due to concerns about losing their competitive advantage or legal compliance issues, whether real or perceived. “A data-sharing approach for greater supply chain viability”[2] by Eleftherios Iakovou and Chelsea C. White III of Brookings depicts this challenge. This challenge has come to the attention of the Biden Administration, as evidenced by the announcement of “Freight Logistics Optimization Works” or FLOW, an information sharing initiative to pilot key freight information exchange between parts of the goods movement supply chain.
Data Quality: ML models require large, high-quality datasets. In shipping logistics, obtaining accurate and comprehensive data can be challenging due to various sources, inconsistent formats, and reliability issues. Data sources in shipping logistics are diverse, which creates challenges for integration and analysis but represents opportunity for one to do it effectively. These sources include:
Automatic Identification System (AIS): a maritime broadcast system that ships use to transmit data such as ship identification, position, course, and cargo information
Vessel Monitoring System (VMS): A system to use communication satellites to track commercial fishing vessels
Port Authorities and Government Data: data on cargo handling, vessel movements, and other maritime activities from governments and authorities
Private Databases: such as Spire Maritime and S&P Global, offer comprehensive maritime databases that include data on vessel tracking, ship movements, and port events
CargoIS (operated by International Air Transport Association (IATA)): data on air cargo movements, offering insights into airfreight charges and other relevant information
Satellite Images: high-resolution images of vessels or coastal areas, which can be used in vessel detection systems and pollution monitoring
Unifying this data, and that like it in other industries categorized by legacy software represents an immense opportunity for value creation. By completing this data jig-saw puzzle, machine learning can identify opportunities for efficiencies, make predictions, optimizing decision making, recognize anomalies, and revolutionize how humans interact with shipping logistics systems, such as through natural language.
Regulation
Beyond the technological barriers associated with data in global shipping logistics, the international nature of the system leads to bureaucratic grime in the gears of the network.
Regulatory and Compliance Issues: The shipping industry is subject to various regulations and compliance standards and ensuring that ML solutions adhere to these regulations can be a significant challenge. These regulatory and compliance issues can be categorized into environmental, sanction/legal, and customs/trade.
Environmental regulations have become of increasing significance as the International Maritime Organization (IMO) and other national governments have implemented stricter rules for the environmental impact of shipping. In fact, the shipping industry is under pressure to reduce carbon intensity by at least 70% by 2050[3].
Sanctions have become an increasingly important tool of global geopolitics and thus increasingly relevant to players in the global shipping system. For instance, during the peak of Russia’s occupation of Ukraine, it was subject to more than 5,000 different targeted sanctions globally from 45 countries at a speed of implementation unimaginable in recent history. In fact, the crippling economic sanctions that targeted Iran were adopted over the course of 10 years; the majority of Russian sanctions have been implemented in just 10 days[4]. Any system must keep pace with this world order as non-compliance can lead to penalties, fines, shipment delays, legal action, and reputational damage.
Customs procedures and compliance standards vary significantly across countries. Data on customs clearance procedures, import duties and taxes, customs valuation, tariff classifications, and trade agreements must be captured by a system and harmonized with a plethora of documentation, including commercial invoices, packing lists, bill of lading, certificates of origin, import licenses, and permits. Today, these standards are so convoluted that large consulting firms exist to support companies in avoiding shipping delays by navigating the regulations and laws.
Security Concerns: Handling sensitive logistics data requires robust security measures and ML systems need to ensure data privacy and protection against potential cyber threats.
Data and privacy concerns is a go-to straw man for any industry reluctant to increase its reliance on technology and this it must be front of mind when considering applying ML to global shipping logistics. The industry handles critical data, including shipment details, customer information, and operational plans, making it vulnerable to various cyber threats such as ransomware attacks, unauthorized access to control systems, supply chain compromises, and data breaches. That said, a recent survey by PwC found that 38% of logistics companies have significant unresolved questions regarding data privacy and security, making it fruitful ground for cyber attacks[5]. Such vulnerabilities and the associated potential operational disruptions, financial losses, and repetitional damage represent a threat to the application of more advance technology to global shipping as they can be used a specious arguments against technological advancement.
Behavior
Costs and Resource Allocation: Implementing ML solutions requires investment in technology, infrastructure, and skilled personnel and thus some companies may face challenges in allocating resources for ML adoption. The capital-intensive nature of the shipping industry also means that successful investments can generate large profits, but conversely, failed investments can lead to substantial losses. This is further complicated by the fact that the shipping industry faces highly volatile freight rates and ship prices. Finally, the free cash necessary to invest in technology without debt and thus increasing leverage is highly dependent on where in the global shipping logistics supply chain a provider is. For instance, the top 25 marine transport businesses report average gross margins between ~20% and ~30% [6] while the average freight forwarding business has single-digit profit margins [7]. This has obvious implications on how innovation can be applies to these various businesses.
Human Resistance to Technology: Resistance from employees or stakeholders to embrace ML-driven changes such as automation of traditional logistics processes can hinder successful implementation. This resistance can stem from discomfort with stepping out of legacy processes and work flows or concerns around job security given human tasks can now be more effectively executed by computers or machines. Despite their limitations, many companies continue to rely on these legacy systems due to familiarity, the high costs and complexities associated with transitioning to new systems, and concerns about disrupting existing workflow. Legacy systems are deeply embedded in the operations and culture of many organizations, and abandoning them can feel like abandoning the business itself. Additionally, employees in the shipping logistics space often fear that AI or bots will take over their jobs, leading to conscious or unconscious resistance to change. This fear is not unfounded, as studies have shown that the introduction of robots in certain sectors has led to job losses. For example, one study found that the addition of one robot replaced about 3.3 workers[8]. That said, technology and labor can often be thought of as “creative destruction”, in that technology creates new needs and employment opportunities through disruption of the existing job market.
Hope in LLMs
I believe there is hope for addressing many of the above barriers in the application of large language models (LLMs). There are a number of distinguished capabilities that are helpful to understand in the context of their potential. Each of these have numerous applications in the context of the barriers to ML applied to the ultimate optimization problem of supply chain and logistics.
Natural Language Understanding and Generation
LLMs have demonstrated remarkable abilities in understanding and generating human language. They can comprehend context, answer questions, write essays, and even create poetry or stories that are often indistinguishable from those written by human.
Application in Shipping Logistics: LLMs can be used to decipher regulatory documents, complete forms and filings, and extract data from rudimentary systems such as paper documents at a human-like accuracy. This removes the need for manual data entry, creating opportunities of automation, and, when applied at scale across the whole value chain of shipping logistics, will improve the overall quality of data by unlocking new data sources and minimizing human error.
In-Context Learning
LLMs are capable of in-context learning, meaning they can understand and respond to instructions given in the context of a conversation or a piece of text. This allows them to adapt to new tasks without the need for additional training.
Application in Shipping Logistics: Shipping is a highly dynamic environment. LLMs ability to resolve exceptions in a process with the benefit of context through knowledge management, such as being trained on internal data such as emails, documents, and transactional history, makes it highly capable of improving customer experience. LLMs can then be the used to automate customer interactions, providing quick and accurate responses to inquiries.
Multimodal Abilities
Although not explicitly mentioned in the search results, LLMs can also be multimodal, meaning they can understand and generate content that combines text with other forms of data like images or sounds, further expanding their application scope.
Application in Shipping Logistics: By interpreting multiple types of input data, such as text, images, audio, and video, simultaneously, LLMs can analyze a variety of data types that are common in the logistics industry, such as textual shipment data, numerical performance metrics, and images from delivery routes. Through this, more data becomes interpretable by a computer, generating more opportunities for optimization and automation.
Planning and Search
LLMs have shown potential in planning tasks, including domain extraction, graph search path planning, and adversarial planning. They can be fine-tuned to improve their capabilities in these areas[9].
Application in Shipping Logistics: LLM-based agents represent a fantastic decision support mechanism for domain experts in shipping logistics. For instance, human experts and interact in natural language with models trained on diverse datasets about global trade networks. Such interactions empower humans to leverage LLM outputs alongside personal expertise during scenario evaluations and policy formulation stages preceding concrete implementation plans.
Collaboration with Smaller Models
Language models (LMs) can work in conjunction with smaller models to enhance performance on supervised tasks, demonstrating their flexibility and ability to integrate with other AI systems[10].
Application in Shipping Logistics: Many complex processes of the global shipping logistics process can be effectively broken up into more simple, discrete tasks. Such tasks can be more easily and accurately completed by a machine learning model. In the context of freight forwarding, Flexport has done this effectively in order to apply LLMs for automation as described by Ryan Peterson in the Dynamo Ventures The Future of Supply Chain Podcast Episode #174.
So What?
The global shipping logistics space is immense, to the point that it is difficult to quantify because it touches nearly every other part of the economy. It is also inefficient, to the tune of hundreds of billions of dollars. LLMs could be the key to applying the ultimate optimization technology to this ultimate optimization problem, unlocking unprecedented economic value and will be further covered in upcoming publications of this newsletter.
—
[1]: Logistics Management Magazine
[2]: Brookings
[3]: SINAY
[4]: Lloyds
[5]: Softlink
[6]: RSM
[7]: GoFreight
[8]: Coupa
[9]: On the Planning, Search, and Memorization Capabilities of Large Language Models
[10]: Small Models are Valuable Plug-ins for Large Language Models