ESIP CCC Knowledge Base¶

Welcome to the Earth Science Information Partners (ESIP) Cloud Computing Cluster (CCC) Knowledge Base!

Here you will find optimization practices and examples for data in the cloud.

Note

🚧 Work in progress!

Note

The Cloud Computing Cluster recognizes that there is no “one-size-fits-all” approach to leveraging the opportunities of the cloud and addressing its challenges. Cluster members enjoy learning and discussing a range of solutions, given any particular use case may have different context and requirements.

Why Optimize Data for Cloud Access?¶

Increasingly, organizations with large data holdings are turning to cloud-hosted storage to improve capacity, scalability, and access to computing resources near data. These data are often stored in object stores or other storage that offers horizontally-scalable access over a network. This demands attention from data providers seeking to maximize the value of data, as the performance characteristics of cloud-hosted stores can differ greatly from their predecessors. Further, the cost profile of cloud hosting differs from that of on premises hosting, such that unnecessary data requests, data transfer, and compute time all tend to directly contribute to the overall cost to host data.

In short, well-optimized data in a cloud environment are typically less expensive, more usable, and have faster typical access times than poorly optimized data.

Cut from cookiecutter-jupyter-book.