View source for Ultra-large-scale systems


'''Ultra-large-scale system''' ('''ULSS''') is a term used in fields including Computer Science, Software Engineering and Systems Engineering to refer to [[Software|software]] intensive systems with unprecedented amounts of [[Computer hardware|hardware]], lines of [[Source code|source code]], numbers of [[User (computing)|users]], and volumes of [[Data|data]]. The scale of these systems gives rise to many problems: they will be developed and used by many [[Social:Stakeholder (corporate)|stakeholders]] across multiple organizations, often with conflicting purposes and needs; they will be constructed from heterogeneous parts with complex dependencies and emergent properties; they will be continuously evolving; and software, hardware and human failures will be the norm, not the exception. The term 'ultra-large-scale system' was introduced by Northrop and others<ref name="ULSSreport">{{cite book |author1=Peter H. Feiler |author2=Kevin Sullivan |author3=Kurt C. Wallnau |author4=Richard P. Gabriel |author5=John B. Goodenough |author6=Richard C. Linger |author7=Thomas A. Longstaff |author8=Rick Kazman |author9=Mark H. Klein |author10=Linda M. Northrop |author11=Douglas Schmidt |editor=Bill Pollak |others=ULS Systems Study Lead: Linda M. Northrop |title=Ultra-large-scale systems: The software challenge of the future |date=2006 |publisher=Software Engineering Institute, Carnegie Mellon University |location=Pittsburgh |isbn=978-0-9786956-0-6 |url=https://resources.sei.cmu.edu/library/asset-view.cfm?assetID=30519}} {{webarchive|url=https://web.archive.org/web/20221128214752/https://resources.sei.cmu.edu/asset_files/Book/2006_014_001_635801.pdf |date=28 November 2022}}</ref> to describe challenges facing the United States Department of Defense. The term has subsequently been used to discuss challenges in many areas, including the computerization of [[Finance:Financial market|financial market]]s.<ref name="GFM brief">{{citation|last1=Cliff |first1=Dave |last2=Northrop |first2=Linda |date=September 2011 |url= https://webarchive.nationalarchives.gov.uk/ukgwa/20140108141316/http://www.bis.gov.uk/foresight/our-work/projects/published-projects/computer-trading/reports-and-publications |title=The Global Financial Markets: An Ultra-Large Scale Systems Perspective |series=The Future of Computer Trading in Financial Markets |id=driver review – DR 4}}. {{webarchive |url=https://web.archive.org/web/20111226070651/http://www.bis.gov.uk/assets/bispartners/foresight/docs/computer-trading/11-1223-dr4-global-financial-markets-systems-perspective.pdf |date=26 December 2011}}. Briefing paper for Government Office for Science, from the [https://www.gov.uk/government/collections/future-of-computer-trading Foresight project: ''Future of computer trading'' (23 October 2012) documents collection]</ref> The term "ultra-large-scale system" (ULSS) is sometimes used interchangeably with the term "large-scale complex IT system" (LSCITS). These two terms were introduced at similar times to describe similar problems, the former being coined in the United States and the latter in the United Kingdom.

==Background==
The term ultra-large-scale system was introduced in a 2006 report from the [[Organization:Software Engineering Institute|Software Engineering Institute]] at [[Organization:Carnegie Mellon University|Carnegie Mellon University]] authored by Linda Northrop and colleagues.<ref name="ULSSreport" /> The report explained that software intensive systems are reaching unprecedented scales (by measures including lines of code; numbers of users and stakeholders; purposes the system is put to; amounts of data stored, accessed, manipulated, and refined; numbers of connections and interdependencies among components; and numbers of hardware elements). When systems become ultra-large-scale, traditional approaches to engineering and management will no longer be adequate. The report argues that the problem is no longer of engineering systems or [[System of systems|system of systems]], but of engineering "[[Sociotechnical system|socio-technical ecosystem]]s".

In 2013, Linda Northrop and her team conducted a talk to review outcome of the 2006 study and the reality of 2013. In summary, the talk concluded that (a) ULS systems are in the midst of society and the changes to current social fabric and institutions are significant; (b) The 2006 original research team was probably too conservative in their report; (c) Recent technologies have exacerbated the pace of scale growth; and (d) There are great opportunities.<ref>{{cite web|last=Northrop|first=Linda|title=Does Scale Really Matter? Ultra-Large-Scale Systems Seven Years After the Study|url=http://resources.sei.cmu.edu/asset_files/Presentation/2013_017_101_68602.pdf|work=Software Engineering Institute, Carnegie Mellon University|access-date=19 May 2014}}</ref>

At a similar time to the publication of the report by Northrop and others, a research and training initiative was being initiated in the UK on [[Organization:Large-scale Complex IT Systems|Large-scale Complex IT Systems]]. Many of the challenges recognized in this initiative were the same as, or were similar to those recognized as the challenges of ultra-large-scale systems.<ref name="sommerville">{{citation |last=Sommerville |first=Ian |title=Ultra Large Scale Systems – Systems Engineering for LSCITS: Lecture presentation slides (York EngD programme) |url=http://www.cs.st-andrews.ac.uk/~ifs/Teaching/Sys%20Eng%20for%20LSCITS/L8-ULSS.pdf |date=2009 |archive-url=https://web.archive.org/web/20120405051938/http://www.cs.st-andrews.ac.uk/~ifs/Teaching/Sys%20Eng%20for%20LSCITS/L8-ULSS.pdf |archive-date=5 April 2012 |publisher=University of St Andrews, School of Computer Science}}</ref> Greg Goth<ref name="goth">{{cite journal |last1=Goth |first1=Greg |title=Ultralarge Systems: Redefining Software Engineering? |journal=IEEE Software |date=May 2008 |volume=25 |issue=3 |pages=91–94 |doi=10.1109/MS.2008.82}}</ref> quotes Dave Cliff, director of the UK initiative as saying "The ULSS proposal and the LSCITS proposal were written entirely independently, yet we came to very similar conclusions about what needs to be done and about how to do it". A difference pointed out by [[Biography:Ian Sommerville (academic)|Ian Sommerville]]<ref name="sommerville" /> is that the UK initiative began with a five to ten year vision, while that of Northrop and her co-authors was much longer term. This seems to have led to there being two slightly different perspectives on ultra-large-scale systems. For example, Richard Gabriel's perspective is that ultra-large-scale systems are desirable but currently impossible to build due to limitations in the fields of software design and systems engineering.<ref name="gabriel">Gabriel, R. P. (16 November 2007). [http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf Design Beyond Human Abilities], Transcript of Wei Lun Public Lecture at the Chinese University of Hong Kong, {{webarchive |url=https://web.archive.org/web/20090423222544/http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf |date=23 April 2009 }}</ref> On the other hand, [[Biography:Ian Sommerville (academic)|Sommerville]]'s perspective is that ultra-large-scale systems are already emerging (for example in [[Engineering:Air traffic control|air traffic control]]), the key problem being not how to achieve them but how to ensure they are adequately engineered.<ref name="sommerville" />

==Characteristics of an ultra-large-scale system==
Ultra-large-scale systems hold the characteristics of systems of systems (systems that have: operationally independent sub-systems; managerially independent components and sub-systems; evolutionary development; emergent behavior; and geographic distribution). In addition to these, the Northrop report<ref name="ULSSreport" /> argues that a ULSS will:
* Have decentralized data, development, evolution and operational control
* Address inherently conflicting, unknowable, and diverse requirements
* Evolve continuously while it is operating, with different capabilities being deployed and removed
* Contain heterogeneous, inconsistent and changing elements
* Erode the people system boundary. People will not just be users, but elements of the system and affecting its overall emergent behavior.
* Encounter failure as the norm, rather than the exception, with it being extremely unlikely that all components are functioning at any one time
* Require new paradigms for acquisition and policy, and new methods for control

The Northrop report<ref name="ULSSreport" /> states that "the sheer scale of ULS systems will change everything. ULS systems will necessarily be decentralized in a variety of ways, developed and used by a wide variety of stakeholders with conflicting needs, evolving continuously, and constructed from heterogeneous parts. People will not just be users of a ULS system; they will be elements of the system. The realities of software and hardware failures will be fundamentally integrated into the design and operation of ULS systems. The acquisition of a ULS system will be simultaneous with its operation and will require new methods for control. In ULS systems, these characteristics will dominate. Consequently, ULS systems will place unprecedented demands on software acquisition, production, deployment, management, documentation, usage, and evolution practices."

==Domains==
The term ultra-large-scale system was introduced by Northrop and others<ref name="ULSSreport" /> to discuss challenges faced by the United States Department of Defense in engineering software intensive systems. In 2008 Greg Goth wrote that although Northrop's report focused on the US military's future requirements, "its description of how the fundamental principles of software design will change in a global economy{{nbsp}}... is finding wide appeal".<ref name="goth" /> The term is now used to discuss problems in several domains.

===Defense===
The Northrop report argued that "the U.S. Department of Defense (DoD) has a goal of information dominance{{nbsp}}... this goal depends on increasingly complex systems characterized by thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through heterogeneous wired and wireless networks.{{nbsp}}...These systems will push far beyond the size of today's systems by every measure{{nbsp}}... They will be ultra-large-scale systems."<ref name="ULSSreport" />

===Financial trading===
Following the [[Finance:2010 Flash Crash|2010 Flash Crash]], Cliff and Northrop<ref name="GFM brief" /> have argued "The very high degree of interconnectedness in the global markets means that entire trading systems, implemented and managed separately by independent organizations, can rightfully be considered as significant constituent entities in the larger global super-system.{{nbsp}}... The sheer number of human agents and computer systems connected within the global financial-markets system-of-systems is so large that it is an instance of an ultra-large-scale system, and that largeness-of-scale has significant effects on the nature of the system".<ref name="GFM brief" />

===Healthcare===
Kevin Sullivan has stated that the US healthcare system is "clearly an ultra-large-scale system"<ref name="sullivan">Sullivan, K. A. (19 August 2011). "Cyber-Social Systems Approach to the Engineering of Ultra-Large-Scale National Health Information Systems". ''Institute of Medicine of the National Academies''. </ref> and that building national scale [[Cyberinfrastructure|cyberinfrastructure]] for healthcare "demands not just a rigorous, modern software and systems engineering effort, but an approach at the cutting edge of our understanding of information processing systems and their development and deployment in complex socio-technical environments".<ref name="sullivan" />

===Others===
Other domains said to be seeing the rise of ultra-large-scale systems include government, transport systems (for example air traffic control systems), energy distribution systems (for example smart grids) and large enterprises.

==Research==
Fundamental gaps in our current understanding of software and software development at the scale of ULS systems present profound impediments to the technically and economically effective achievement of significant gains in core system functionality. These gaps are strategic, not tactical. They are unlikely to be addressed adequately by incremental research within established categories. Rather, we require a broad new conception of both the nature of such systems and new ideas for how to develop them. We will need to look at them differently, not just as systems or systems of systems, but as socio-technical ecosystems. We will face fundamental challenges in the design and evolution, orchestration and control, and monitoring and assessment of ULS systems. These challenges require breakthrough research.<ref name="ULSSreport" />

===In the United States===
The Northrop report proposed that a portfolio of interdisciplinary research be developed, following a ULS systems research agenda that highlights the following areas:<ref name="ULSSreport" />
Human interaction – People are key participants in ULS systems. Many problems in complex systems today stem from failures at the individual and organizational level. Understanding ULS system behavior will depend on the view that humans are elements of a socially constituted computational process. This research involves anthropologists, [[Social:Sociologist|sociologist]]s, and social scientists conducting detailed socio-technical analyses of user interactions in the field, with the goal of understanding how to construct and evolve such socio-technical systems effectively.<ref>{{Cite web |url=http://www.sei.cmu.edu/uls/research/interaction/|title=Ultra-Large-Scale Systems {{!}} Human Interaction |archive-date=6 October 2009 |archive-url=https://web.archive.org/web/20091006055700/http://www.sei.cmu.edu/uls/research/interaction/ |website=Software Engineering Institute |publisher=Carnegie Mellon University}}</ref>

Computational emergence – ULS systems must satisfy the needs of participants at multiple levels of an organization. These participants will often behave opportunistically to meet their own objectives. Some aspects of ULS systems will be "programmed" by properly incentivizing and constraining behavior rather than by explicitly prescribing. This research area explores the use of methods and tools based on economics and [[Game theory|game theory]] (for example, mechanism design) to ensure globally optimal ULS system behavior by exploiting the strategic self-interests of the system's constituencies. This research area also includes exploring [[Metaheuristic|metaheuristic]]s and digital evolution to augment the cognitive limits of human designers, so they can manage ongoing ULS system adaptation more effectively.<ref>{{cite web |title=Ultra-Large-Scale Systems {{!}} Computational Emergence |url=http://www.sei.cmu.edu/uls/research/emergence/|website=Software Engineering Institute |publisher=Carnegie Mellon University |archive-date=6 October 2009 |archive-url=https://web.archive.org/web/20091006055644/http://www.sei.cmu.edu/uls/research/emergence/}}</ref>

Design – Current design theory, methods, notations, tools, and practices and the acquisition methods that support them are inadequate to design ULS systems effectively. This research area broadens the traditional technology-centric definition of design to include people and organizations; social, cognitive, and economic considerations; and design structures such as design rules and government policies. It involves research in support of designing ULS systems from all of these points of view and at many levels of abstraction, from the hardware to the software to the people and organizations in which they work.<ref>{{Cite web|url=http://www.sei.cmu.edu/uls/research/design/|title=Ultra-Large-Scale Systems {{!}} Design |archive-url=https://web.archive.org/web/20091006055639/http://www.sei.cmu.edu/uls/research/design/ |archive-date=6 October 2009 |website=Software Engineering Institute |publisher=Carnegie Mellon University }}</ref>

Computational engineering – New approaches will be required to enable intellectual control at an entirely new level of scope and scale for system analysis, design, and operation. ULS systems will be defined in many languages, each with its own abstractions and semantic structures. This research area focuses on evolving the expressiveness of representations to accommodate this semantic diversity. Because the complexity of ULS systems will challenge human comprehension, this area also focuses on providing automated support for computing the behavior of components and their compositions in systems and for maintaining desired properties as ULS systems evolve.<ref>{{cite web |website=Software Engineering Institute |publisher=Carnegie Mellon University |url=http://www.sei.cmu.edu/uls/research/engineering/|title=Ultra-Large-Scale Systems {{!}} Computational Engineering |archive-date=6 October 2009 |archive-url=https://web.archive.org/web/20091006055649/http://www.sei.cmu.edu/uls/research/engineering/ }}</ref>

Adaptive system infrastructure – ULS systems require an infrastructure that permits organizations in distributed locations to work in parallel to develop, select, deploy, and evolve system components. This research area investigates integrated development environments and runtime platforms that support the decentralized nature of ULS systems. This research also focuses on technologies, methods, and theories that will enable ULS systems to be developed in their deployment environments.<ref>{{Cite web |url=http://www.sei.cmu.edu/uls/research/infrastructure/|title=Ultra-Large-Scale Systems {{!}} Adaptive System Infrastructure |website=Software Engineering Institute |publisher=Carnegie Mellon University |archive-date=6 October 2009 |archive-url=https://web.archive.org/web/20091006055656/http://www.sei.cmu.edu/uls/research/infrastructure/}}</ref>

Adaptable and predictable system quality – ULS systems will be long-running and must operate robustly in environments fraught with failures, overloads, and [[Cyberattack|cyberattack]]s. These systems must maintain robustness in the presence of adaptations that are not centrally controlled or authorized.<ref>{{Cite web |website=Software Engineering Institute |publisher=Carnegie Mellon University |url=http://www.sei.cmu.edu/uls/research/quality/|title=Ultra-Large-Scale Systems {{!}} Adaptable and Predictable System Quality |archive-date=6 October 2009 |archive-url= https://web.archive.org/web/20091006055715/http://www.sei.cmu.edu/uls/research/quality/ }}</ref>

Managing traditional qualities such as [[Computer security|security]], performance, reliability, and [[User interface|usability]] is necessary but not sufficient to meet the challenges of ULS systems. This research area focuses on how to maintain quality in a ULS system in the face of continuous change, ongoing failures, and attacks. It also includes identifying, predicting, and controlling new indicators of system health (akin to the U. S. [[Finance:Gross domestic product|gross domestic product]]) that are needed because of the scale of ULS systems.

Policy, acquisition, and management – Policy and management frameworks for ULS systems must address organizational, technical, and operational policies at all levels. Rules and policies must be developed and automated to enable fast and effective local action while preserving global capabilities. This research area focuses on transforming acquisition policies and processes to accommodate the rapid and continuous evolution of ULS systems by treating suppliers and [[Supply chain|supply chain]]s as intrinsic and essential components of a ULS system.<ref>{{Cite web |url=http://www.sei.cmu.edu/uls/research/policy/ |title=Ultra-Large-Scale Systems {{!}} Policy, Acquisition, and Management |website=Software Engineering Institute |publisher=Carnegie Mellon University |archive-date=6 October 2009 |archive-url=https://web.archive.org/web/20091006055710/http://www.sei.cmu.edu/uls/research/policy/}}</ref>

The proposed research does not supplant current, important software research but rather significantly expands its horizons. Moreover, because it is focused on systems of the future, the SEI team purposely avoided couching descriptions in terms of today's technology. The envisioned outcome of the proposed research is a spectrum of technologies and methods for developing these systems of the future, with national-security, economic, and societal benefits that extend far beyond ULS systems themselves.

===In the United Kingdom===
The UK's research programme in [[Organization:Large-scale Complex IT Systems|Large-scale Complex IT Systems]]<ref>{{cite web|url=http://www.lscits.org|title=Welcome|website=[[Organization:Large-scale Complex IT Systems|Large-scale Complex IT Systems]]}} {{Webarchive|url=https://web.archive.org/web/20111123190211/http://lscits.cs.bris.ac.uk/|date=23 November 2011}}</ref><ref>{{cite web |title=Projects – LSCITS-RPV2: Large-scale Complex IT Systems Initiative: Overview, Effective start→end date – 1 July 2007 → 1 July 2013 |url=https://research-information.bris.ac.uk/en/projects/lscits-rpv2-large-scale-complex-it-systems-initiative |website=University of Bristol |publisher=Department of Computer Science; Cabot Institute for the Environment; Intelligent Systems Laboratory; Systems Centre |language=en}} {{Webarchive |url=https://web.archive.org/web/20220819000515/https://research-information.bris.ac.uk/en/projects/lscits-rpv2-large-scale-complex-it-systems-initiative|date=19 August 2022}}</ref> has been concerned with issues around ULSS development and considers that an LSCITS (Large-scale complex IT system) shares many of the characteristics of a ULSS.

===In China===
The National Natural Science Foundation of China has outlined a five-year project for researchers to study the assembly of ultra-large spacecraft.<ref>{{cite web |url= https://spacenews.com/china-researching-challenges-of-kilometer-scale-ultra-large-spacecraft |title= China researching challenges of kilometer-scale ultra-large spacecraft |date= Aug 27, 2021 |website= Space News |last= Jones |first= Andrew }}</ref> Although vague, the project would have applications for potential [[Megaproject|megaproject]]s, including colossal [[Physics:Space-based solar power|space-based solar power]] stations. Work on an Ultra-Large Aperture On-Orbit Assembly Project under the Chinese Academy of Sciences (CAS) and with support from the Chinese Ministry of Science and Technology is already underway.

==See also==
* [[Complex adaptive system]]
* [[Philosophy:Emergence|Emergence]]
* [[IT portfolio management]]
* [[Operator overloading]]
* [[Physics:Self-organization|Self-organization]]
* Sociotechnical systems
* [[Software architecture]]
* [[Systems design]]
* [[Systems theory]]

==References==
{{Reflist}}

==Further reading==
* {{cite web |last1=Northrop |first1=Linda M. |title=Does Scale Really Matter? Ultra-Large-Scale Systems Seven Years After the Study |url=https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=68596 |website=Software Engineering Institute |publisher=Carnegie Mellon University |archive-url=https://web.archive.org/web/20220212204918/https://resources.sei.cmu.edu/asset_files/Presentation/2013_017_101_68602.pdf |archive-date=12 February 2022 |language=en |date=24 May 2013 |id=Keynote presented at the 35th International Conference on Software Engineering (ICSE 2013) and published by IEEE Computer Society Press}}
* Peter Van Roy (2008). [https://www.info.ucl.ac.be/~pvr/vanroy-mc-panel.pdf "The Challenges and Opportunities of Multiple Processors: Why Multi-Core Processors are Easy and Internet is Hard"] [http://lambda-the-ultimate.org/node/2874 – via ''Lambda the Ultimate'']. Discussion paper that touches on topics important in ULS research.
* Chapters from: {{cite book |editor=B. A. Huberman |title=The ecology of computation |date=1988 |publisher=Elsevier Science North Holland |location=Amsterdam |isbn=978-0444703750}} Three foundational papers on capability-based market-oriented computing (concepts that are the subject of some ULS Systems research), by [[Biography:Mark S. Miller|Mark S. Miller]] and [[Biography:K. Eric Drexler|K. Eric Drexler]] [https://papers.agoric.com/papers/ – via The Agoric Papers] — ([https://web.archive.org/web/20071217095249/http://www.agorics.com/Library/agoricpapers.html archived index]): 
** {{cite book |last1=Miller |first1=Mark S. |last2=Drexler |first2=K. Eric |chapter=Markets and Computation: Agoric Open Systems |title = The Ecology of Computation |date=1988 |pages=133–176 |chapter-url=https://papers.agoric.com/assets/pdf/papers/markets-and-computation-agoric-open-systems.pdf |archive-date=14 August 2022 |archive-url=https://web.archive.org/web/20220814073327/https://papers.agoric.com/assets/pdf/papers/markets-and-computation-agoric-open-systems.pdf |display-authors=0}}
** {{cite book |last1=Miller |first1=Mark S. |last2=Drexler |first2=K. Eric |chapter = Incentive Engineering for Computational Resource Management |title = The Ecology of Computation |date=1988 |pages=231–266 |chapter-url=https://papers.agoric.com/assets/pdf/papers/incentive-engineering-for-computational-resource-management.pdf |archive-date=5 December 2022 |archive-url=https://web.archive.org/web/20221205125848/https://papers.agoric.com/assets/pdf/papers/incentive-engineering-for-computational-resource-management.pdf |display-authors=0}}
** {{cite book |last1=Miller |first1=Mark S. |last2=Drexler |first2=K. Eric |chapter = Comparative ecology: A computational perspective |title = The Ecology of Computation |date=1988 |pages=51–76 |chapter-url=https://papers.agoric.com/assets/pdf/papers/comparative-ecology-a-computational-perspective.pdf |archive-date=24 January 2023 |archive-url=https://web.archive.org/web/20230124052108/https://papers.agoric.com/assets/pdf/papers/comparative-ecology-a-computational-perspective.pdf |display-authors=0}}

[[Category:Computer engineering]]
[[Category:Computer systems]]
[[Category:Electronic design automation]]
[[Category:Systems analysis]]
[[Category:Systems engineering]]
[[Category:Systems theory]]

{{Sourceattribution|Ultra-large-scale systems}}