Please use this identifier to cite or link to this item: http://hdl.handle.net/1959.3/196660

Download PDF (Accepted manuscript) (Adobe Acrobat PDF, -1 bytes)

Title

A local-optimisation based strategy for cost-effective datasets storage of scientific applications in the cloud

Author(s)

Yuan, Dong;  Yang, Yun;  Liu, Xiao;  Chen, Jinjun

Abstract

Massive computation power and storage capacity of cloud computing systems allow scientists to deploy computation and data intensive applications without infrastructure investment, where large application datasets can be stored in the cloud. However, due to the pay-as-you-go model, the datasets should be strategically stored in order to reduce the overall application cost. In this paper, by utilising Data Dependency Graph (DDG) from data provenances in scientific applications, deleted datasets can be regenerated, and as such we develop a novel cost-effective datasets storage strategy that can automatically store appropriate datasets in the cloud. This strategy achieves a localised optimal trade-off between computation and storage, meanwhile also taking users' tolerance of data accessing delay into consideration. Simulations conducted on general (random) datasets and a specific astrophysics pulsar searching application with Amazon's cost model show that our strategy can reduce the application cost significantly.

Publication type

Conference paper

Research centre

Swinburne University of Technology. Faculty of Information and Communication Technologies

Source

Proceedings of the 4th IEEE International Conference on Cloud Computing (CLOUD 2011), Washington, District of Columbia, United States, 04-09 July 2011 / Ling Liu and Manish Parashar (eds.), pp. 179-186

Publication year

2011

FOR Code(s)

0806 Information Systems

Keyword(s)

Cloud computing;  Computation-storage trade-off;  Datasets storage;  Scientific applications

Publisher

IEEE

ISSN

2159-6182 (series ISSN)

ISBN

9781457708367, 1457708361

Publisher URL

http://dx.doi.org/10.1109/CLOUD.2011.13

Copyright

Copyright © 2011 IEEE. The accepted manuscript of the paper is reproduced here in accordance with the copyright policy of the publisher. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

Research Projects

Cost effective storage of massive intermediate data in cloud computing applications, Australian Research Council grant number DP110101340

Full text

Peer reviewed