Public Cloud Storage and Database Review (May 2020 update)

Alfredo Surroca Database, Storage

GCP Database and Storage Services

GCP Enabling hybrid storage with a third party product called Egnyte.
Egnyte, a Google Cloud Technology Partner and a 2016 Gartner Magic Quadrant Leader for Enterprise File Synchronization and Sharing (EFSS), offers Google customers the ability to create a hybrid SaaS file sync and share infrastructure that harnesses the power and flexibility of Google Cloud services with the security and centralized IT administration of on-premises content management.

 

AWS Database and Storage Services

Amazon also offers some additional services like:

  • Amazon Timestream
    A fast, scalable, fully managed time series database service for IoT and operational applications that makes it easy to store and analyze trillions of events per day at 1/10th the cost of relational databases.
  • Amazon Quantum Ledger Database (QLDB)
    That provides a high-performance, immutable, cryptographically verifiable ledger for applications where multiple parties work with a centralized, trusted authority to maintain a complete, verifiable record of transactions.
  • Amazon Keyspaces (for Apache Cassandra)
    A scalable, highly available, and managed Apache Cassandra–compatible database service.
  • Amazon DynamoDB Accelerator (DAX)
    Aa fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement – from milliseconds to microseconds – even at millions of requests per second.
  • Amazon FSx for Lustre
    A fully managed file system that is optimized for compute-intensive workloads, such as high performance computing, machine learning, and media data processing workflows, and is seamlessly integrated with Amazon S3
  • Amazon FSx for Windows File Server
    A fully managed native Microsoft Windows file system built on Windows Server so you can easily move your Windows-based applications that require file storage to AWS, including full support for the SMB protocol and Windows NTFS, Active Directory (AD) integration, and Distributed File System (DFS).
  • AWS Storage Gateway
    A hybrid cloud storage service that gives you on-premises access to virtually unlimited cloud storage. The service provides three different types of gateways – Tape Gateway, File Gateway, and Volume Gateway

 

Azure Database and Storage Services

Azure has the following additional database and storage services:

  • Azure SQL Database Edge (preview)
    A smallfootprint, edge-optimized, and AI-capable data engine built on the same code base as Microsoft Azure SQL Database and Microsoft SQL Server.
    Currently in preview, SQL Database Edge simplifies IoT infrastructure by offering streaming, storage, and analytics all in one platform.
  • Azure Storage Tables (replaced by Azure Cosmos DB Table API)
    A key-value database solution with rows and columns. Tables store data as a collection of entities where each entity has a property. Azure Tables can have up to 255 properties (columns in relational databases). The maximum entity size (row size in a relational database) is 1 MB.
  • Azure Queue storage
    A service for storing large numbers of messages that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. A single queue message can be up to 64 KB in size, and a queue can contain millions of messages, up to the total capacity limit of a storage account.
  • Azure Data Lake Storage
    A set of capabilities dedicated to big data analytics, built on Azure Blob storage
  • Azure File Sync, HPC Cache & Data Box (Hybrid Storage)
    You can use Azure File Sync to centralize your organization’s file shares in Azure Files, while keeping the flexibility, performance, and compatibility of an on-premises file serve.
    Azure Data Box Gateway is a storage solution that enables you to seamlessly send data to Azure.
    Azure HPC Cache speeds access to your data for high-performance computing (HPC) tasks. This service can be used even for workflows where your data is stored across WAN links, such as in your local datacenter network-attached storage (NAS) environment.
  • Avere vFXT for Azure & Azure FXT Edge Filer (hybrid storage cache)
    Avere vFXT for Azure is a filesystem caching solution for data-intensive high-performance computing (HPC) tasks. It lets you take advantage of cloud computing’s scalability to make your data accessible when and where it’s needed – even for data that’s stored in your own on-premises hardware.
    Azure FXT Edge Filer is a hybrid storage caching appliance that provides fast file access and active archive for high-performance computing (HPC) tasks.

In addition, just to mention, that Azure under Cosmos DB is the only Cloud provider that offer a multi-model database service to three of the Non Relational Databases;  Key-Value, Document and Graph  with SQL Api access.

 

Alibaba Database and Storage Services

Alibaba also offers some additional services like:

  • Time Series Database (TSDB)
    As a highly reliable and cost-effective time-series database that provides efficient data reading and writing capabilities, high-compression-ratio storage, and time-series data interpolation and aggregation. TSDB has wide industrial applications including Internet of Things (IoT) monitoring systems, enterprise-level energy management systems (EMS), production safety monitoring, and electric power detection systems.
  • Cloud Storage Gateway
    A gateway service that you can deploy on the premises or in the cloud. This service provides a seamless and secure connection between your on-premises IT infrastructure and cloud-based storage services at the back end.
  • Hybrid Cloud Storage Array (coming soon)    
    A cost-effective, efficient and easy-to-manage hybrid cloud storage solution.
  • Hybrid Backup Recovery
    An enterprise-level data backup and migration service that is secure, efficient, cost-effective, and scalable. Hybrid Backup Recovery provides full protection for data stored in Alibaba Cloud and on-premises data centers.

Database & Storage Migration

In general, there are two approaches to migrate your legacy Database to any of the Cloud DB Services:

  • Migrating to the same type of database
    • You can lift and shift your database to the Market managed database offered by the Cloud Provider
    • You can lift and shift your database to the Market managed database offered by third party (MongoDB Atlas)
    • You can lift and shift your database to the IaaS offered by the Cloud Provider
  • Migrating to a new type of database offered by the Cloud Provider

In addition, you have to review/modify the database calls in the source code. It is an occasion to return to the old concepts of IO modules to isolate the accesses to the database and facilitate its future migration to new database and cloud implementations.

Related to storage, the approach is to offer a Storage Transfer service to move data from legacy storage or another Cloud Provider Storage.

Google Database & Storage Migration guides and utilities

Google Cloud offers a migration assessment guide, migration tools (mainly data transfer tools), and collaboration with our partners to help manage the full life cycle of database migration https://cloud.google.com/db-migration/.

Assessment Guides:

Relate to Storage, Google offers a Storage Transfer Service that allows you to quickly import online data into Cloud Storage. You can also set up a repeating schedule for transferring data, as well as transfer data within Cloud Storage, from one bucket to another.

Finally for hybrid approach,  Egnyte, a Google Cloud Technology Partner, offers Google customers the ability to create a hybrid SaaS file sync and share infrastructure that harnesses the power and flexibility of Google Cloud services with the security and centralized IT administration of on-premises content management. https://cloud.google.com/solutions/partners/egnyte-enabling-hybrid-storage

In addition there is a guide about how to migrate from Amazon Simple Storage Service (Amazon S3) to Cloud Storage.

Amazon Database & Storage Migration guides and utilities

Amazon offers a more sophisticate approach to migrate legacy Databases to cloud (https://aws.amazon.com/dms/). In addition to guides and partners they offer tools like:

  • AWS Schema Conversion Tool
    That makes heterogeneous database migrations predictable by automatically converting the source database schema and a majority of the database code objects, including views, stored procedures, and functions, to a format compatible with the target database.
  • AWS Database Migration Service (AWS DMS)
    A cloud service that makes it easy to migrate relational databases, data warehouses, NoSQL databases, and other types of data stores. You can use AWS DMS to migrate your data into the AWS Cloud, between on-premises instances (through an AWS Cloud setup), or between combinations of cloud and on-premises setup.

Related to Storage, Amazon offers a suite of tools to help you move data via networks, roads and technology partners. Amazon offers two kind of migration tools:

  • Unmanaged Cloud Data Migration Tools
    • rsync.
    • S3 command line interface.
    • Glacier command line interface.
  • Managed Cloud Data Migration Tools
    • AWS Direct Connect
    • AWS Snowball Edge
    • AWS Snowmobile
    • Amazon Transfer Family

Finally AWS offers an Hybrid approach for storage called AWS Storage Gateway that connects an on-premises software appliance with cloud-based storage to provide seamless integration with data security features between your on-premises IT environment and the AWS storage infrastructure. AWS Storage Gateway offers file-based, volume-based, and tape-based storage solutions.

Azure Database & Storage Migration guides and utilities

Azure offers a Database Migration Service (https://azure.microsoft.com/en-us/services/database-migration/) that enables seamless migrations from multiple database sources to Azure Data platforms with minimal downtime. The service uses the Data Migration Assistant to generate assessment reports that provide recommendations to guide you through the changes required prior to performing a migration. When you’re ready to begin the migration process, the Azure Database Migration Service performs all of the required steps.

The options covered by the service are:

  • Migrate SQL Server to Azure SQL Database
  • Migrate MySQL to Azure Database for MySQL
  • Migrate PostgreSQL to Azure Database for PostgreSQL
  • Migrate MongoDB to Azure Cosmos DB Mongo

Related to Storage, Azure like AWS offers a suite of tools to help you move data via networks, roads and technology partners.

  • Unmanaged Cloud Data Migration Tools
    • AzCopy
    • Azure PowerShell
    • Azure CLI
    • Azure Storage REST APIs/SDKs.
  • Managed Cloud Data Migration Tools
    • Azure Data Box family for online transfers
    • Azure Data Factory
    • Azure Import/Export

In addition, Azure has incremented his portfolio of Hybrid Storage approach with:

  • Azure File Sync, HPC Cache & Data Box (Hybrid Storage)
    You can use Azure File Sync to centralize your organization’s file shares in Azure Files, while keeping the flexibility, performance, and compatibility of an on-premises file serve.
    Azure Data Box Gateway is a storage solution that enables you to seamlessly send data to Azure.
    Azure HPC Cache speeds access to your data for high-performance computing (HPC) tasks. This service can be used even for workflows where your data is stored across WAN links, such as in your local datacenter network-attached storage (NAS) environment.
  • Avere vFXT for Azure & Azure FXT Edge Filer (hybrid storage cache)
    Avere vFXT for Azure is a filesystem caching solution for data-intensive high-performance computing (HPC) tasks. It lets you take advantage of cloud computing’s scalability to make your data accessible when and where it’s needed – even for data that’s stored in your own on-premises hardware.
    Azure FXT Edge Filer is a hybrid storage caching appliance that provides fast file access and active archive for high-performance computing (HPC) tasks.

Alibaba Database & Storage Migration guides and utilities

Alibaba offers specific services to migrate Databases to his Cloud and also offers a Alibaba Cloud Data Transmission Service (DTS) to achieve object definition and data migration. Object definition migration means migrating the definition syntax of structure objects, such as a table or a view, to the target database. In addition to the object definition, DTS migrates data stored in the table to the target database.

The options covered by DTS are:

  • On-premises databases to RDS instances or ECS instance databases
  • ECS instance databases to RDS instances
  • Redis instances to Redis instances in classic networks or ECS instance databases

Related to Storage, Alibaba (apart from DTS) offers OssImport to migrate data from third-party storage products (or from another OSS source like Amazon S3) to OSS.

Finally, Alibaba like his peers has started to develop hybrid approaches like:

  • Cloud Storage Gateway
    A gateway service that you can deploy on the premises or in the cloud. This service provides a seamless and secure connection between your on-premises IT infrastructure and cloud-based storage services at the back end.
  • Hybrid Cloud Storage Array (coming soon)    
    A cost-effective, efficient and easy-to-manage hybrid cloud storage solution.
  • Hybrid Backup Recovery
    An enterprise-level data backup and migration service that is secure, efficient, cost-effective, and scalable. Hybrid Backup Recovery provides full protection for data stored in Alibaba Cloud and on-premises data centers.

Bonus – MongoDB Atlas

Mongo DB Atlas is the most relevant multi-cloud managed database. Choose from over 60 cloud regions across Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP).

You can also move any deployment to MongoDB Atlas with minimal impact to your application using the live migration feature in the application. Live migration works by keeping a cluster in MongoDB Atlas in sync with your source database until you’re ready to cut over.

MongoDB Atlas takes care of:

  • Automates infrastructure provisioning, setup, and deployment
  • Assure high availability
  • Handle backups
  • Secure my data
  • Monitoring and performance optimization
  • Support for the cloud database platform and product usage inquiries
  • SLA

You can review a comparison of Mongo DB Atlas  with the managed services of Amazon DocumentDB and Azure Cosmos DB API; https://www.mongodb.com/cloud/atlas/compare

Pages: 1 2 3