Why You Need A Federated Software System

To date, most enterprise systems targeted at specific industries have been designed as centralized systems, rather than those with open protocols and a federated model. Adopting a federated model for a systems architecture is typically deemed to have too much complexity and risk. This has, since the advent of computing, funneled organizations into buying proprietary software from controlled silos which do not afford flexibility and portability for their critical data, and which inherently ultimately undermines their autonomy. If an industry is particularly fertile ground for IT system innovation, they may see multiple competing vendors carving out kingdoms of clients; this is the current scenario of least harm, and usually the only options in front of a procurement officer tasked with buying software for their enterprise. How can you retain your autonomy as a business, and still exchange and interact with others in your industry network? The answer is: Federation.

For smaller, less lucrative industries with lower liquidity (such as social work, as one example), there may only be one or two viable candidate systems to be purchased. This lack of competition will drive up prices, discourage innovation in the systems through tepid competitive dynamics, and create an unfavorable power differential between the de facto system and their grudging clients. This toxic relationship forces client organizations to either give up their autonomy (at a high per-seat cost), or else it encourages them to cobble together spreadsheets and sticks and try to make it on their own - usually much to the detriment of efficiency in the practice of their actual business.

As new systems and architectures have been laid overtop the old over the past 5 decades, and stable federated systems have become more achievable and the best practices for their implementation explored, it is now possible for certain industries to adopt federated software systems. There are some industries that are an especially good fit: Any industry with a loosely bound network of independent agencies or organizations, but who need to perform data sharing of some type, are ones that are particularly primed for a federated information system.

There are several famous information systems already in use that are built according to federated principles: DNS, Email, the World Wide Web itself, all created in the 1970s and 1980s. But these relatively naive implementations came with hurdles because of their federated model and they have themselves trended towards centralization over time. This is why most of web traffic consists of users posting content to one of four big social media platforms, and why email, for example, which was designed as a highly federated system has centralized down into the hands of just a few very large players (Gmail, and Outlook.com, et al).

Federated Systems suffer from several challenges:

Communication: Federated networks are potentially comprised of a massive number of devices (e.g., millions of smart phones), and communication in the network can be slower than local computation by many orders of magnitude.

Complexity: Federated architecture is expected to deliver high flexibility and agility among independently cooperating components. However, it can instead often increase complexity and make it unmanageable.

Security: Federated systems are more vulnerable to security breaches than centralized systems because they have more entry points for attackers.

Data privacy: Federated systems require data sharing between semi-autonomous de-centrally organized lines of business (LOBs), information technology systems and applications. This can lead to data privacy concerns.

Many of these problems are nonexistent in their centralized alternatives, although there are (of course) other issues with them. Federation problems are simply harder to solve in a technical way, and have resisted technical solutions for a longer time. Since there is a monoplistic benefit attached to controlling a silo, there has been relatively little incentive for software creators to move away from centralized offerings into federated and federable systems. Early attempts at solving these issues manifested in non-commercial P2P filesharing networks like Napster, Gnutella and Bittorrent in the early 2000s, and then with increasing complexity and sophistication in blockchain technology in even more recent times - all seeking to answer federation challenges with technical solutions. In the wake of the upheavals following Elon Musk’s acquisition of Twitter, two federated social media alternatives have lately gained prominence and much attention: Mastodon and Bluesky; both of which are federated (or federable). Over the past fifty years we have seen a steady maturation of the techniques and tools for creating increasingly sophisticated, performant, and distributed federable systems that are still secure and provide data privacy at scale.

Within the context of software architecture, a federated model has the potential to reshape power differentials between individual parties involved in the system. By its very nature, a federated approach distributes power, authority, and decision-making across the participating entities or components. Unlike a centralized model, where a single governing entity holds significant control, a federated model disperses responsibilities and allows for greater autonomy among the parties involved.

Health organizations often see clients and client data being moved from provider to provider, whether that movement is due to the patient picking up and moving to another geographical location, or to the fact that they must move between various specialists to receive appropriate treatment. If the health data for those patients is centrally held, then it is not held in trust by the doctor’s offices and mental health counselling agencies, and the hospitals and housing workers and everyone else; it is held by a third party acting as the governing entity.

Federated systems can also be used to share production data between different manufacturers participating in a complicated supply chain, ensuring that each manufacturer controls their own outputs to the greater product while still allowing for the collaboration that complex products require. In retail industries, it is often desireable to share customer data between different stores so they can provide a common rewards program and support each other’s business, and in finance it is critical for institutions to be able to integrate with each other so that money can move around as frictionlessly as possible.

In a secure federated system, each institution or organization maintains control over its own systems and data, but also establishes connections and interfaces with other entities in the network. Their own data is resident on their own systems, never inaccessible or owned by another, never viewable by another without explicit consent, and only shared to network partners in the federation when it is proper to do so. They have the freedom to make independent decisions pertaining to their specific domain or component, within the agreed-upon boundaries and protocols established by the federation. This decentralization of power can empower individual parties to have more agency and influence over the aspects they control. Different systems, applications, and institutions can operate independently while adhering to shared standards and protocols. This enables the exchange of information, transactions, and services in a standardized and coordinated manner.

Furthermore, a federated model encourages collaboration and cooperation among the participating parties. Instead of relying solely on a central authority, decision-making is distributed and shared. This collaborative aspect promotes a more egalitarian environment, where parties have the opportunity to contribute their expertise and influence the direction of the system. This can help mitigate power imbalances that may arise in a more traditional, hierarchical software architecture. Obviously this is preferable, from the standpoint of sovereignty, to handing over decisionmaking to an arbitrating centralized system.

However, it is important to note that power differentials can still exist within a federated model, albeit in a different form. The influence and control exerted by each party may vary based on factors such as their resources, expertise, or the criticality of their component to the overall system. While the federated model aims to distribute power more equitably, it does not guarantee absolute parity among all parties. For example, when Microsoft refused to support otherwise shared standards in rendering web pages with their powerfully overrepresented Internet Explorer browser, this had the consequence of fracturing web development and making it more complicated for decades, due to the need to support multiple, often mutually incompatible, variant standards. It is crucial to establish effective governance mechanisms, clear communication channels, and well-defined protocols to ensure fairness and balance within the federated system.

In summary, a federated model for software architecture has the potential to reshape power differentials by decentralizing authority, promoting collaboration, and providing more autonomy to individual parties. While it strives to distribute power more equitably, careful attention must be given to governance and communication to ensure a balanced and inclusive environment for all participants.

When it is time for you to choose software for your business, and you are faced with the usual “Sophie’s Choice” between two entitled software vendors peddling exclusively centralized systems, consider the option of joining or developing a federated, decentralized option instead. These may exist already in the form of an open source software system, many of which have integrators ready to help you adopt them, or they may be something you will need to build out yourself according to emerging ISO standards or government regulation applicable to your industry. But they are an option that won’t see you locked into a proprietary centralized system with no sovereign control, it will ultimately lift up others in your industry network, and you’ll always be able to secure and transform your own data.