Ultra-large-scale systems

Ultra-large-scale system (ULSS) is a term used in fields including Computer Science, Software Engineering and Systems Engineering to refer to software intensive systems with unprecedented amounts of hardware, lines of source code, numbers of users, and volumes of data. The scale of these systems gives rise to many problems: they will be developed and used by many stakeholders across multiple organizations, often with conflicting purposes and needs; they will be constructed from heterogeneous parts with complex dependencies and emergent properties; they will be continuously evolving; and software, hardware and human failures will be the norm, not the exception. The term 'ultra-large-scale system' was introduced by Northrop and others to describe challenges facing the United States Department of Defense. The term has subsequently been used to discuss challenges in many areas, including the computerization of financial markets. The term "ultra-large-scale system" (ULSS) is sometimes used interchangeably with the term "large-scale complex IT system" (LSCITS). These two terms were introduced at similar times to describe similar problems, the former being coined in the United States and the latter in the United Kingdom.

Background

The term ultra-large-scale system was introduced in a 2006 report from the Software Engineering Institute at Carnegie Mellon University authored by Linda Northrop and colleagues. The report explained that software intensive systems are reaching unprecedented scales (by measures including lines of code; numbers of users and stakeholders; purposes the system is put to; amounts of data stored, accessed, manipulated, and refined; numbers of connections and interdependencies among components; and numbers of hardware elements). When systems become ultra-large-scale, traditional approaches to engineering and management will no longer be adequate. The report argues that the problem is no longer of engineering systems or system of systems, but of engineering "socio-technical ecosystems".

In 2013, Linda Northrop and her team conducted a talk to review outcome of the 2006 study and the reality of 2013. In summary, the talk concluded that (a) ULS systems are in the midst of society and the changes to current social fabric and institutions are significant; (b) The 2006 original research team was probably too conservative in their report; (c) Recent technologies have exacerbated the pace of scale growth; and (d) There are great opportunities.

At a similar time to the publication of the report by Northrop and others, a research and training initiative was being initiated in the UK on Large-scale Complex IT Systems. Many of the challenges recognized in this initiative were the same as, or were similar to those recognized as the challenges of ultra-large-scale systems. Greg Goth quotes Dave Cliff, director of the UK initiative as saying "The ULSS proposal and the LSCITS proposal were written entirely independently, yet we came to very similar conclusions about what needs to be done and about how to do it". A difference pointed out by Ian Sommerville is that the UK initiative began with a five to ten year vision, while that of Northrop and her co-authors was much longer term. This seems to have led to there being two slightly different perspectives on ultra-large-scale systems. For example, Richard Gabriel's perspective is that ultra-large-scale systems are desirable but currently impossible to build due to limitations in the fields of software design and systems engineering. On the other hand, Sommerville's perspective is that ultra-large-scale systems are already emerging (for example in air traffic control), the key problem being not how to achieve them but how to ensure they are adequately engineered.

Characteristics of an ultra-large-scale system

Ultra-large-scale systems hold the characteristics of systems of systems (systems that have: operationally independent sub-systems; managerially independent components and sub-systems; evolutionary development; emergent behavior; and geographic distribution). In addition to these, the Northrop report argues that a ULSS will:

• Have decentralized data, development, evolution and operational control
• Address inherently conflicting, unknowable, and diverse requirements
• Evolve continuously while it is operating, with different capabilities being deployed and removed
• Contain heterogeneous, inconsistent and changing elements
• Erode the people system boundary. People will not just be users, but elements of the system and affecting its overall emergent behavior.
• Encounter failure as the norm, rather than the exception, with it being extremely unlikely that all components are functioning at any one time
• Require new paradigms for acquisition and policy, and new methods for control

The Northrop report states that "the sheer scale of ULS systems will change everything. ULS systems will necessarily be decentralized in a variety of ways, developed and used by a wide variety of stakeholders with conflicting needs, evolving continuously, and constructed from heterogeneous parts. People will not just be users of a ULS system; they will be elements of the system. The realities of software and hardware failures will be fundamentally integrated into the design and operation of ULS systems. The acquisition of a ULS system will be simultaneous with its operation and will require new methods for control. In ULS systems, these characteristics will dominate. Consequently, ULS systems will place unprecedented demands on software acquisition, production, deployment, management, documentation, usage, and evolution practices."

Domains

The term ultra-large-scale system was introduced by Northrop and others to discuss challenges faced by the United States Department of Defense in engineering software intensive systems. In 2008 Greg Goth wrote that although Northrop's report focused on the US military's future requirements, "its description of how the fundamental principles of software design will change in a global economy ... is finding wide appeal". The term is now used to discuss problems in several domains.

Defense

The Northrop report argued that "the U.S. Department of Defense (DoD) has a goal of information dominance ... this goal depends on increasingly complex systems characterized by thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through heterogeneous wired and wireless networks. ...These systems will push far beyond the size of today's systems by every measure ... They will be ultra-large-scale systems."

Financial trading

Following the 2010 Flash Crash, Cliff and Northrop have argued "The very high degree of interconnectedness in the global markets means that entire trading systems, implemented and managed separately by independent organizations, can rightfully be considered as significant constituent entities in the larger global super-system. ... The sheer number of human agents and computer systems connected within the global financial-markets system-of-systems is so large that it is an instance of an ultra-large-scale system, and that largeness-of-scale has significant effects on the nature of the system".

Healthcare

Kevin Sullivan has stated that the US healthcare system is "clearly an ultra-large-scale system" and that building national scale cyberinfrastructure for healthcare "demands not just a rigorous, modern software and systems engineering effort, but an approach at the cutting edge of our understanding of information processing systems and their development and deployment in complex socio-technical environments".

Others

Other domains said to be seeing the rise of ultra-large-scale systems include government, transport systems (for example air traffic control systems), energy distribution systems (for example smart grids) and large enterprises.

Research

Fundamental gaps in our current understanding of software and software development at the scale of ULS systems present profound impediments to the technically and economically effective achievement of significant gains in core system functionality. These gaps are strategic, not tactical. They are unlikely to be addressed adequately by incremental research within established categories. Rather, we require a broad new conception of both the nature of such systems and new ideas for how to develop them. We will need to look at them differently, not just as systems or systems of systems, but as socio-technical ecosystems. We will face fundamental challenges in the design and evolution, orchestration and control, and monitoring and assessment of ULS systems. These challenges require breakthrough research.

In the United States

The Northrop report proposed that a portfolio of interdisciplinary research be developed, following a ULS systems research agenda that highlights the following areas: Human interaction – People are key participants in ULS systems. Many problems in complex systems today stem from failures at the individual and organizational level. Understanding ULS system behavior will depend on the view that humans are elements of a socially constituted computational process. This research involves anthropologists, sociologists, and social scientists conducting detailed socio-technical analyses of user interactions in the field, with the goal of understanding how to construct and evolve such socio-technical systems effectively.

Computational emergence – ULS systems must satisfy the needs of participants at multiple levels of an organization. These participants will often behave opportunistically to meet their own objectives. Some aspects of ULS systems will be "programmed" by properly incentivizing and constraining behavior rather than by explicitly prescribing. This research area explores the use of methods and tools based on economics and game theory (for example, mechanism design) to ensure globally optimal ULS system behavior by exploiting the strategic self-interests of the system's constituencies. This research area also includes exploring metaheuristics and digital evolution to augment the cognitive limits of human designers, so they can manage ongoing ULS system adaptation more effectively.

Design – Current design theory, methods, notations, tools, and practices and the acquisition methods that support them are inadequate to design ULS systems effectively. This research area broadens the traditional technology-centric definition of design to include people and organizations; social, cognitive, and economic considerations; and design structures such as design rules and government policies. It involves research in support of designing ULS systems from all of these points of view and at many levels of abstraction, from the hardware to the software to the people and organizations in which they work.

Computational engineering – New approaches will be required to enable intellectual control at an entirely new level of scope and scale for system analysis, design, and operation. ULS systems will be defined in many languages, each with its own abstractions and semantic structures. This research area focuses on evolving the expressiveness of representations to accommodate this semantic diversity. Because the complexity of ULS systems will challenge human comprehension, this area also focuses on providing automated support for computing the behavior of components and their compositions in systems and for maintaining desired properties as ULS systems evolve.

Adaptive system infrastructure – ULS systems require an infrastructure that permits organizations in distributed locations to work in parallel to develop, select, deploy, and evolve system components. This research area investigates integrated development environments and runtime platforms that support the decentralized nature of ULS systems. This research also focuses on technologies, methods, and theories that will enable ULS systems to be developed in their deployment environments.

Adaptable and predictable system quality – ULS systems will be long-running and must operate robustly in environments fraught with failures, overloads, and cyberattacks. These systems must maintain robustness in the presence of adaptations that are not centrally controlled or authorized.

Managing traditional qualities such as security, performance, reliability, and usability is necessary but not sufficient to meet the challenges of ULS systems. This research area focuses on how to maintain quality in a ULS system in the face of continuous change, ongoing failures, and attacks. It also includes identifying, predicting, and controlling new indicators of system health (akin to the U. S. gross domestic product) that are needed because of the scale of ULS systems.

Policy, acquisition, and management – Policy and management frameworks for ULS systems must address organizational, technical, and operational policies at all levels. Rules and policies must be developed and automated to enable fast and effective local action while preserving global capabilities. This research area focuses on transforming acquisition policies and processes to accommodate the rapid and continuous evolution of ULS systems by treating suppliers and supply chains as intrinsic and essential components of a ULS system.

The proposed research does not supplant current, important software research but rather significantly expands its horizons. Moreover, because it is focused on systems of the future, the SEI team purposely avoided couching descriptions in terms of today's technology. The envisioned outcome of the proposed research is a spectrum of technologies and methods for developing these systems of the future, with national-security, economic, and societal benefits that extend far beyond ULS systems themselves.

In the United Kingdom

The UK's research programme in Large-scale Complex IT Systems has been concerned with issues around ULSS development and considers that an LSCITS (Large-scale complex IT system) shares many of the characteristics of a ULSS.

In China

The National Natural Science Foundation of China has outlined a five-year project for researchers to study the assembly of ultra-large spacecraft. Although vague, the project would have applications for potential megaprojects, including colossal space-based solar power stations. Work on an Ultra-Large Aperture On-Orbit Assembly Project under the Chinese Academy of Sciences (CAS) and with support from the Chinese Ministry of Science and Technology is already underway.

See also

• Complex adaptive system
• Emergence
• IT portfolio management
• Operator overloading
• Self-organization
• Sociotechnical systems
• Software architecture
• Systems design
• Systems theory

Further reading

• Northrop, Linda M. (24 May 2013). "Does Scale Really Matter? Ultra-Large-Scale Systems Seven Years After the Study" (in en). Carnegie Mellon University. https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=68596. 
• Peter Van Roy (2008). "The Challenges and Opportunities of Multiple Processors: Why Multi-Core Processors are Easy and Internet is Hard" – via Lambda the Ultimate. Discussion paper that touches on topics important in ULS research.
• Chapters from: B. A. Huberman, ed (1988). The ecology of computation. Amsterdam: Elsevier Science North Holland. ISBN 978-0444703750.  Three foundational papers on capability-based market-oriented computing (concepts that are the subject of some ULS Systems research), by Mark S. Miller and K. Eric Drexler – via The Agoric Papers — (archived index):
  • Miller, Mark S. et al. (1988). "Markets and Computation: Agoric Open Systems". The Ecology of Computation. pp. 133–176. https://papers.agoric.com/assets/pdf/papers/markets-and-computation-agoric-open-systems.pdf. 
  • Miller, Mark S. et al. (1988). "Incentive Engineering for Computational Resource Management". The Ecology of Computation. pp. 231–266. https://papers.agoric.com/assets/pdf/papers/incentive-engineering-for-computational-resource-management.pdf. 
  • Miller, Mark S. et al. (1988). "Comparative ecology: A computational perspective". The Ecology of Computation. pp. 51–76. https://papers.agoric.com/assets/pdf/papers/comparative-ecology-a-computational-perspective.pdf.