Cloud Cost Comparison : 1vm/mo

 In this post, we will explore and compare the cost of hosting a small VM for 1 month in different clouds. 

1. Serverspace

https://serverspace.us/services/cloud-servers/
Pricing Ref:         https://serverspace.us/pricing/
Configuration: Ubuntu 18.04, 1GB, 1core, 50Mbps, 25GB SSD
Cost/mo:         USD 4.55
Cost/hr:              USD 0.006

2. Kamatera Cloud

https://www.kamatera.com/express/compute/
Configuration: Ubuntu 18.04, 1GB, 1core, 50Mbps, 20GB SSD
Cost/mo:         USD 4.00
Cost/hr:                 USD 0.005
Pricing Ref:         https://www.kamatera.com/express/compute/?tcampaign=35187_368194&bta=35187&nci=5344

3. Linode

https://www.linode.com
Configuration: 1GB, 1core, 25GB SSD, 1TB, 40GB NW In, 1000Mbps NW Out
Cost/mo:         USD 5.00
Cost/mo:         USD 0.0075
Pricing Ref:         https://www.linode.com/pricing/

4. ScalaHosting

https://www.scalahosting.com/cloud-servers
Configuration: Ubuntu 20.04, 2GB, 1core, 50GB SSD, 3000GB bandwidth
Cost/mo:         USD 10.00
Cost/hr:                 USD 0.015
Pricing Ref:         https://my.scalahosting.com/order.php?a=configure&i=0

5. Cloudways

https://www.cloudways.com/
Using there interface you only can create VMs in other cloud provideres, such as DigitalOcean, Linode, VULTR, AWS, Google Cloud
Pricing Ref: https://www.cloudways.com/en/pricing.php

6. LiquidWeb

https://www.liquidweb.com/products/cloud-dedicated/
Configuration: Linux, 2GB, 2 vCPU, 40GB SSD, 10TB bandwidth
Cost/mo:             USD 15 (if you go for 24months plan)
Other Addon:        InterWorx, Plesk Web Pro, or cPanel Admin
Ref:                         https://www.liquidweb.com/products/vps/

7. SiteGround

https://www.siteground.com/cloud-hosting.htm
Configuration: 8GB, 4cores, 40GB SSD, 5TB data transfer
Cost/mo:         USD 80.00
Pricing Ref:         https://www.siteground.com/cloud-hosting.htm?afcode=b0dbbe40d137d9fab687993c2f1ce093&campaign=Cloud+Hosting

8. DigtialOcean

https://www.digitalocean.com/
Configuration: 1GB, 1CPU, 25GB SSD, 1TB data transfer
Cost/mo:         USD 5.00
Cost/hr:                 USD 0.00744
Pricing Ref:         https://www.digitalocean.com/pricing/

9. Vultr

https://www.vultr.com/
Configuration: 512MB, 1CPU, 10GB SSD, 0.5TB data transfer
Cost/mo:         USD 3.5 (can be 2.5USD if you choose IPv6 version only )
Cost/hr:                 USD 0.005
Pricing Ref:         https://www.vultr.com/products/cloud-compute/#pricing

10. Amazon Web Services

https://aws.amazon.com/
Configuration: 1GB, 2vCPU, 10GB HDD, 0.5TB data transfer
Cost/mo:         USD 4.02 
Cost/hr:                 USD 0.005
Pricing Ref:         https://calculator.aws/#/createCalculator/EC2  

Integrating Apache Nifi with Azure Storage

 In this post we will see how Apache Nifi can be used to handle the blobs/files in Azure Blob storage.

We will go through:

• Creating a Storage account in Microsoft Azure
• Creating Nifi template that download the files/blobs from Azure blob storage
• Creating a Nifi template that upload files to the blob storage
• Creating a Nifi template that delete blobs/files from the blob storage

Prerequisites:

It is expected that, you have fundamental knowledge on 

• Microsoft Azure
• Cloud computing
• Apache Nifi
• Data pipeline

Steps:

1. Create the Storage Account

1. Go to Azure home portal: https://portal.azure.com/?quickstart=True#home 

2. Click on Storage Account :

3. Click on Add Button, if you don’t have a Storage account. Below figures are for your reference:

4. Now go to the Storage Account portal: https://portal.azure.com/?quickstart=True#blade/HubsExtension/BrowseResource/resourceType/Microsoft.Storage%2FStorageAccounts 

5. Click on the storage account name

6.  Under “Blob service” section

1. Click on Containers.

2. Click + Container button to create a new container

3. Give a name and leave others to default

4. Now click on the Create button at the bottom. 

5. Now click on container name 

6. Click on Upload button to upload a file

7. Now you need to get the Access Keys

1. Under Settings section, click on Access Keys 

2. Now click on Show Keys button

3. Note down the Key (not connection string) under key1

2. Nifi template for downloading the blobs from a container

8. Go to Nifi interface

9. Create a ListAzureBlobStorage processor.

10. Configure with following properties

1. Container Name: Get the name you have created in previous step

2. Storage Credentials:

1. Create a new Controller Services “AzureStorageCredentialsControllerService”

2. Properties are: Storage Account name (same as you have created before), Storage Account Key (same as you have noted down in previous step)

3. Now Enable the Controller Service 

11. Create a FetchAzureBlobStorage processor with the same properties of  ListAzureBlobStorage processor. 

12. Create a PutFile processor, with the following properties:

1. Directory: /tmp/azure/success (you can change this according to your requirement)

2. Leave rest properties to default

13. Create a PutFile processor, with the following properties. This is just to track if anything goes wrong:

1. Directory: /tmp/azure/fail (you can change this according to your requirement)

2. Leave rest properties to default

14. Now the Nifi template should look like below:

3. Nifi template to UPLOAD files to a container

15. Get the following processors and connect them.

16. Properties of GetFile:

17. Properties of PutAzureBlobStorage processor:

4. Nifi template to delete files/blob from a container

18. Create the following nifi processor:

19. Properties of ListAzureBlobStorage processor:

20. Properties of DeleteAzureBlobStorageprocessor:

Here I am deleting only one blob(i.e. Image 2.png) . But you can leave the Blob properties to its default value to delete all the blobs

Virtualization: The technology behind cloud

Virtualization is the dominant technology used to set up cloud environment. In traditional approach, without using virtualization technology, only one OS can be installed onto one physical machine. However, using virtualization technology, multiple machines can be created, called Virtual machines (VMs) or virtual server. In research articles, PMs are also referred to physical servers. Similarly, the physical resources are also refers to substrate resource. Now, lets understand the figure given below.

Fig: Type-1 and Type-2 hypervisor

[src]: https://commons.wikimedia.org/wiki/File:Hyperviseur.png#/media/File:Hyperviseur.png

The virtual machines can be created only with the help of a small piece of software call virtual machine monitor (VMM) or virtual manager that implements the virtualization technology. The VMM can be installed directly on the hardware (Type 1 in the above figure) or can be installed atop an OS (Type 2 in the above figure). When the VMM is installed atop an OS, that OS is called hosted OS and the OS that is installed in each virtual machine is called guest OS. This virtualization layer provides and environment for the VMs to interact with the hardware. Another major responsibility of virtualization layer is to maintain the isolation among the VMs or guest OSs.

Here question arises, how this technology helps us to setup a cloud environment. When the cloud user ask for a computer with the configuration 2GB memory, 2 processing units, 50GB storage, and 2mbps network connection, cloud service provider creates a VM with the configuration provided by the cloud user. The cloud user is than given a URL in order to establish a connection to the created VM. Cloud user has no information regarding the underlined PM upon which the VM is created. In general CSPs adopt the TYPE 1 (in the above figure) approach to create the VMs. multiple VMs are created upon single PM. Another major advantage of this virtualization technology is that, it is very easy to migrate a VM fromone PM to another. Users are also NOT given any facility to interact with the physical machine which make the PMs more secure from the outside attacker.

Here is the link to calculate the number of VMs per PM or Host. In today's date an average of 40-60 VMs are created on each PM.

Based on the where the VMM is installed and purpose of the installation, virtualization is of four types^[1].

    1. Hardware Virtualization : If the VMM is install atop the hardware, e,i, Type 1 in the above figure

    2. OS Virtualization : If the VMM is install atop an Operating System, e.i. Type 2 in the above Figure

    3. Storage Virtualization : Here the VMM is installed on multiple storage servers, which forms a single storage system for the user.

    4. Server Virtualization: When the VMM installed to divide the single physical server system into multiple virtual servers.

Besides aforemention four types, virtualization can also be divided into data virtualization, memory virtualization, network virtualization etc categories^[2].

Thats all about the basics of Virtualization. Please comment if you need more further detail information.

Thank you

Cloud computing for layman

Before digging to the concept of cloud computing, Let's see how we consume electricity. Basically in every region some companies/organisations are providing electricity to every house. Electricity provider uses Copper or Aluminium wire to provide eletricity to our houses. In addition to that, external electric meter is attached in every house to monitor the amount of electric power we are consuming.

Now imagine computer instead of electricity. Some organization may offer you computers instead of electricity. Computers will be rented to you over high speed internet by some organization. The computers are not provided physically by shifting the machines to your location, rather, the organizations/providers allow you to connect to that computer over high speed internet. A large number of computers are used by the provider. Those large number of computers, also known as servers, are kept in secure place called Data Center. Henceforth, we will use the term server instead of computer. The organization is called cloud service provider (CSP) and the users are called cloud user. So here we have two entities: cloud consumer/user, and the CSP.

The CSP also provide the software on a rented instead of the server. Let’s say, I want to use MS office. In that case I’ll have two options: either buy the software or rent from the CSP. In later option, you will be allowed to access the software through a webpage or some explicitly designed different interface. Such interface will provide you all the required functionalities and you need to pay based on different costing model. This kind of service offered by the CSP is called Software as a Service (SaaS). For example, Google Apps, Salesforce etc

User may also request the CSP multiple number of servers with certain configuration. For example, One may ask the CSP to provide two servers with 4GB memory, 50GB storage, 4 processing units. Offering such service to the cloud users is called Infrastructure as a Service (IaaS). For example Amazon AWS, Windows Azure, Google Compute Engine, IBM SmartCloud Enterprise, Rackspace Open Cloud etc.

Coming to another type of service, CSP also customize the service based on the users requirement such as providing the platform. Here user refers to the software developer. CSP provides all the required software components, such as database servers, programming environment etc, in order to develop a software tool. Such kind of service is called Platform as a Service, in short PaaS. For example, Google App Engine provides the environment to develop web applications using Java, Python, PHP and Go.

Fig. 1 : Snap of data center’s inside view [src]

Thats all about the basics of Cloud Computing. Next, we will see the underlined technology behind the cloud computing.

-Thank you