CELAR

CELAR was a research project which successfully developed an open source set of tools designed to provide automatic, multi-grained resource allocation for cloud applications. In this way CELAR developed a solution that competes directly with Ubuntu Juju, Openstack Heat and Amazon Web Services. CELAR was developed with funding from the European Commission under the Seventh Framework Programme for Research and Technological Development, sometimes abbreviated to FP7.

Partners

CELAR was a noteworthy example of a collaborative research project supported by the European Union involving a number of European partners including the ATHENA Research and Innovation Center (Greece), Flexiant (United Kingdom ), PlayGen Limited (United Kingdom ), SixSq (Switzerland ) the University of Cyprus (Cyprus), the Vienna University of Technology (Austria), The University of Manchester (United Kingdom ) and The Greek Research and Technology Network (Greece).

Summary

The vision of the CELAR project was to provide automatic, multi-grained resource allocation for cloud applications. This enabled the commitment of just the right amount of resources based on application demand, performance and requirements, results in optimal use of infrastructure resources and significant reductions in administrative costs.

The outcome of the CELAR project is an open-source toolkit; a set of tools that allows you to automatically, elastically scale your application deployments. Using the CELAR Platform, you the user can perform the following steps:

• Define deployments of their complex multi-tier distributed application
• Specify the application specific elasticity constraints and actions to fulfil them
• Provide the application custom monitoring probes, provision the application on Clouds with one button click
• Then you can let the CELAR Platform manage and control the scalability lifecycle domain of the application.

CELAR features a c-Eclipse (Eclipse GUI for defining the application’s topology, deployment and elasticity constraints. User application deployments can be defined in any scripting language with an ability to incorporate any of the configuration management systems (Chef, Puppet etc.).

Research

During the lifetime of the project a number of scientific papers were published showcasing the innovation and research of the CELAR consortium:

[C1] Automated, Elastic Resource Provisioning for NoSQLClusters Using TIRAMOLA (May 14, 2013) Best Paper Award, D. Tsoumakos, I. Konstantinou, C. Boumpouka, S. Sioutas and N. Koziris (ATHENA), CCGrid 2013

[C2] SYBL: an Extensible Language for Controlling Elasticity in Cloud Applications (May 13–15, 2013), G. Copil, D. Moldovan, H.-L. Truong, S. Dustdar (TUW), CCGrid 2013

[C3] On Estimating Actuation Delays in Elastic Computing Systems (May 20, 2013), A. Gambi, D. Moldovan, G. Copil, H.-L. Truong, S. Dustdar(TUW), SEAMS 2013

[C4] COCCUS: Self-Configured Cost-Based Query Services in the Cloud (June 22–27, 2013), I. Konstantinou, D. Tsoumakos, and N. Koziris (ATHENA), 2013 ACM SIGMOD/PODS International Conference on Management of Data

[C5] Multi-level Elasticity Control of Cloud Services (short paper) (December 2–5, 2013), G. Copil, D. Moldovan, H.-L. Truong, S. Dustdar (TUW), CSOC 2013

[C6] SYBL+MELA: Specifying, Monitoring, and Controlling Elasticity of Cloud Services (demo paper) (December 2–5, 2013), G. Copil, D. Moldovan, H.-L. Truong, S. Dustdar (TUW), ICSOC 2013

[C7] MELA: Monitoring and Analyzing Elasticity of Cloud Services (2–5 December 2013), D. Moldovan, G. Copil, H.-L. Truong, S. Dustdar (TUW), CloudCom 2013

[C8] CoMoT – A Platform-as-a-Service for Elasticity in the Cloud (10–14 March 2014), Hong-Linh Truong, Schahram Dustdar, Georgiana Copil, AlessioGambi, Waldemar Hummer, Duc-Hung Le, Daniel Moldovan(TUW), Future of PaaS 2014

[C9] JCatascopia: Monitoring Elastically Adaptive Applications in the Cloud, D. Trihinas and G. Pallis and M. D. Dikaiakos, 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID2014)

[C10] Managing and Monitoring Elastic Cloud Applications, D. Trihinas and C. Sofokleous and N. Loulloudes and A.Foudoulis and G. Pallis and M. D. Dikaiakos, 14th International Conference on Web Engineering (ICWE 2014) Poster

[C11] c-Eclipse: An Open-Source Management Framework for Cloud Applications, C. Sofokleous and N. Loulloudes and D. Trihinas and G. Pallisand M. Dikaiakos, EuroPar 2014

[C12] On Controlling Cloud Services Elasticity in Heterogeneous Clouds, Georgiana Copil, Daniel Moldovan, Hong-Linh Truong, Schahram Dustdar, 6th Cloud Control Workshop, 7th IEEE/ACM International Conference on Utility and Cloud Computing, 8–11 December, London, 2014

[C13]QUELLE – a Framework for Accelerating the Development of Elastic Systems, Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, Third European Conference on Service-Oriented and Cloud Computing – ESOCC 2014, 2–4 September, Manchester, United Kingdom

[C14] ADVISE – a Framework for Evaluating Cloud Service Elasticity Behavior (3–6 November 2014) Best Paper Award Georgiana Copil, Demetris Trihinas, Hong-Linh Truong, Daniel Moldovan, George Pallis, Schahram Dustdar, Marios Dikaiakos, 12th International Conference on Service Oriented Computing. Paris, France

[C15] On Analyzing Elasticity Relationships of Cloud Services, (15–18 December 2014) Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, 6th IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2014, Singapore

[C16] SALSA: a Framework for Dynamic Configuration of Cloud Services, (15–18 December 2014) Duc-Hung Le, Hong-Linh Truong, Georgiana Copil, Stefan Nastic and Schahram Dustdar, 6th International Conference on Cloud Computing Technology and Science, CloudCom 2014, Singapore

[C17] Coordination-aware Elasticity, (8–11 December 2014) Stefano Mariani, Hong-Linh Truong, Georgiana Copil, Andrea Omicini, Schahram Dustdar, 7th IEEE/ACM International Conference on Utility and Cloud Computing, London, 2014

[C18] CELAR: Automated Application Elasticity Platform, (27–30 October) Ioannis Giannakopoulos, Nikolaos Papailiou, Christos Mantas, Ioannis Konstantinou, Dimitrios Tsoumakos and Nectarios Koziris, 2014 IEEE International Conference on Big Data, Washington DC, USA, 2014

[C19] Dependable Horizontal Scaling Based On Probabilistic Model Checking, A. Naskos, E. Stachtiari, A. Gounaris, P. Katsaros, D.Tsoumakos, I. Konstantinou and S. Sioutas, CCGrid 2015 conference

[C20] PANIC: Modeling Application Performance over Virtualized Resources, I. Giannakopoulos, D. Tsoumakos, N. Papailiou and N. Koziris, 2015 IEEE International Conference on Cloud Engineering (IC2E 2015)

[C21] I/O Performance Modeling for Big Data Applications over Cloud Infrastructures, I. Mytilinis, D. Tsoumakos, V. Kantere, A. Nanos and N. Koziris, 2015 IEEE International Conference on Cloud Engineering (IC2E 2015)

[C22] Transforming Vertical Web Applications Into Elastic Cloud Applications, Nikola Tankovic, Tihana Galinac Grbac, Hong-Linh Truong, Schahram Dustdar, International Conference on Cloud Engineering (IC2E 2015), 9–12 March 2015, USA, doi:10.1109/IC2E.2015.15

[C23] On Developing and Operating of Data Elasticity Management Process, Tien-Dung Nguyen, Hong-Linh Truong, Georgiana Copil, Duc-Hung Le, Daniel Moldovan, Schahram Dustdar, 13th International Conference on Service Oriented Computing (ICSOC) 2015, 16–19 November, Goa, India

[C24] iCOMOT – Toolset for Managing IoT Cloud Systems, Hong-Linh Truong, Georgiana Copil, Schahram Dustdar, Duc-Hung Le, Daniel Moldovan, Stefan Nastic, 16th IEEE International Conference on Mobile Data Management, 15–18 June 2015, Pittsburg, USA. (Demo), 16th IEEE International Conference on Mobile Data Management, IEEE MDM2015, 15 - 18 June, 2015, Pittsburgh, Pennsylvania, USA

[C25] Programming Elasticity and Commitment in Dynamic Processes, Pablo Fernandez, Hong-Linh Truong, Schahram Dustdar, Antonio Ruiz-Cortes, IEEE Internet Computing, Volume 19, Number 2, pp. 68 – 74, doi:10.1109/MIC.2015.45

[C26] Principles for Engineering IoT Cloud Systems, Hong-Linh Truong, Schahram Dustdar, IEEE Cloud Computing, Volume 2, Issue 2, pp. 68 – 76, doi:10.1109/MCC.2015.23

[C27] Supporting Cloud Service Operation Management for Elasticity, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, 13th International Conference on Service Oriented Computing (ICSOC) 2015, 16–19 November, Goa, India

[C28] Enabling Interoperable Cloud Application Management through an Open Source Ecosystem, N. Loulloudes, C. Sofokleous, D. Trihinas, M. D. Dikaiakos, G. Pallis, IEEE Internet Computing 19(3): 54-59 (2015)

[C29] Enabling Cloud Application Portability, D. Antoniades, N. Loulloudes, A. Foudoulis, C. Sophokleous, D. Trihinas, G. Pallis, M. Dikaiakos, H. Kornmayer, Proceedings of the Cloud Challenge 2015, in conjunction with 8th IEEE/ACM International Conference on Utility and Cloud Computing (UCC), December 7–10, 2015, Limassol, Cyprus.

[C30] CELAR: Automatic, Multi-grained Elasticity Provisioning for the Cloud, M Dikaiakos N. Loulloudes, G. Pallis, H-L. Truong, D. Tsoumakos, 8th IEEE/ACM International Conference on Utility and Cloud Computing (UCC), December 7–10, 2015, Limassol, Cyprus

[C31] Cloud Application Management Framework (CAMF) Tutorial, N. Loulloudes, 2nd Workshop on Cloud Computing in Cyprus: Opportunities and Challenges, University of Cyprus, 3 June 2015, Nicosia, Cyprus

[C32] Enabling Interoperable Cloud Application Management through an Open Source Ecosystem, Nicholas Loulloudes and Chrystalla Sofokleous and Demetris Trihinas and Marios D. Dikaiakos and George Pallis, IEEE Internet Computing Volume 19, Pages: 54–59, 2015

[C33] Monitoring Elastically Adaptive Multi-Cloud Services, D. Trihinas, G. Pallis, M.D Dikaiakos, IEEE Transactions on Cloud Computing In Revision (under second round review)

[C34] AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices, D. Trihinas, G. Pallis, M.D Dikaiakos, 2015 IEEE International Conference on Big Data (IEEE BigData 2015), October, Santa Clara, USA, 2015

[C35] Analysing Cancer Genomics in the Elastic Cloud, Christopher Smowton, Andoena Balla, Demetris Antoniades, Crispin Miller, GeorgePallis, Marios D. Dikaiakos, Wei Xing, The 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, doi:10.1109/CCGrid.2015.176, May 2015, Shengzheng, China

[C36] A Network Approach for Managing and Processing Big Omic Data in Clouds, Wei Xing, Wei Jie, Dimitrios Tsoumakos, Moustafa Ghanem, Journal of Cluster Computing, Special issue on Big Data Computing. doi:10.1007/s10586-015-0456-6, Print ISSN 1386-7857, Online ISSN 1573-7543

[C37] A Cost-Effective Approach to Improving Performance of Big Genomic Data Analyses in Clouds, Christopher Smowton, Andoena Balla, Demetris Antoniades, Crispin Miller, GeorgePallis, Marios D. Dikaiakos, Wei Xing, Submitted to Journal of Future Generation Computer Systems (under review)

[C38] SCAN: A Smart Application Platform for Empowering Parallelization of Big Genomic Data Analysis in Clouds, Wei Xing, Jie Wei, Crispin Miller, 44th International Conference on Parallel Processing (ICPP-2015), 2015, Beijing, China

[C39] Genome Analysis in a Dynamically Scaled Hybrid Cloud, Chris Smowton, Georgiana Copil, Hong-Linh Truong, Crispin Miller and Wei Xing, IEEE eScience 2015, Munich, Germany

[C40] A Cloud-based Data Network Approach for Translational Cancer Research, Wei Xing, Dimitrios Tsoumakos, Moustafa Ghanem, GeNeDis 2014 Computational Biology and Bioinformatics, Springer International Publishing, ISBN:978-3-319-09011-5; DOI: 10.1007/978-3-319-09012-2, 2015

[J1] MELA: Elasticity Analytics for Cloud Services, Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, to appear in International Journal of Big Data Intelligence 2015, Vol. 2, No. 1, International Journal of Big Data Intelligence (IJBDI) Inderscience Publishers - linking academia, business and industry through research

[J2] Evaluating cloud service elasticity behaviour, G. Copil and D. Trihinas and H.L Truong and D. Moldovan and G. Pallis and S. Dustdar and M. D. Dikaiakos, International Journal of Cooperative Information Systems 2015.

[J3]: Programming Elasticity in the Cloud, Hong-Linh Truong, Schahram Dustdar, Computer, March 2015

[BC1] On Controlling Elasticity of Cloud Applications in CELAR, Georgiana Copil, Daniel Moldovan, Duc-Hung Le, Hong-Linh Truong, Schahram Dustdar, Chrystalla Sofokleous, Nicholas Loulloudes, Demetris Trihinas, George Pallis, Marios D. Dikaiakos, Craig Sheridan, Evangelos Floros, Christos KK Loverdos, Kam Star, Wei Xing, to appear in Emerging Research in Cloud Distributed Computing Systems, Advances in Systems Analysis, Software Engineering, and High Performance Computing (ASASEHPC) Book Series

Key: C – Conference, BC – Book Chapter, J – Journal

Background

Auto Scaling Resources is one of the top obstacles and opportunities for cloud computing: consumers can minimize the execution time of their tasks without exceeding a given budget. Cloud providers maximise their financial gain while keeping their customers satisfied and minimizing administrative costs. Many systems claim to offer adaptive elasticity, yet the “throttling” is usually performed manually, requiring the user to figure out the proper scaling conditions. In order to harvest the benefits of elastic provisioning, it is imperative that it be performed in an automated, fully customizable manner. CELAR delivers a fully automated and highly customisable system for elastic provisioning of resources in cloud computing platforms.

CELAR | Future Internet Assembly | Athens 2014 Craig Sheridan, Head of Research Flexiant said in a statement:

“Combining our consultancy and expertise of cloud orchestration with world class academic and industrial partners in this project will offer the consortium the qualitative and quantitative information necessary to gauge platform and application performance to facilitate intelligent deployment decisions.”

Objective

The goal of the CELAR project was to develop methods and open-source tools for applying and controlling multi-grained, elastic resource provisioning for Cloud applications in an automated manner. This resource allocation is to be performed through intelligent decision-making based on:

(a) Cloud and application performance metrics collected and cost-evaluated through a scalable monitoring system and exposed to the user. (b) Qualitative and quantitative characterisation of the application‘s performance through modelling of its elastic properties.

Nectarios Koziris, Project Coordinator and Associate Professor of the School of Electrical and Computer Engineering at the National Technical University of Athens explains:

“The result of CELAR is a software package that offers organisations the right amount of resources based on application demand, performance and requirements resulting in optimal use of infrastructure resources and significant reductions in administrative costs.”

CELAR covers the three layers required by an application to operate over the Cloud:

• Infrastructure layer
• Monitoring/optimization middleware (automatic elasticity provisioning over cloud platforms and multi-layer monitoring)
• Programming development environment (through a distributed tool to enable developers, administrators and users to define the characteristics of their applications, deploy them, and monitor performance).

The outcome is a modular, completely open-source system that offers elastic programmability for the user and automatic elasticity at the platform level. This outcome can be bundled in a single software package for one-click installation of any application alongside its automated resource provisioning over a Cloud IaaS.

Two exemplary applications that showcase and validate the aforementioned technology will be developed: The first will showcase the use of CELAR technology for massive data management and large-scale collaboration required in the on-line gaming realm, while the second will focus on the area of scientific computing, requiring compute- and storage-intensive genome computations.

The CELAR consortium – under the lead of ATHENA Research and Innovation Center – achieved its objectives by bringing together a team of leading researchers in the large-scale technologies such as Cloud/Grid Computing, service-oriented architectures, virtualization, analytics, Web 2.0 and the world of the Semantic Web. These are combined with active industrial and leading user organizations that offer expertise in the cloud application domain and production-level service provisioning.