Spot instances with Linode

A VPS CRUD with Linode Cloud Hosting and Ruby

As a Rails developer have you ever wondered how to launch and delete cloud instance on demand with Ruby?

In this post I will take you through how you can do a CRUD(Create, Read, Update, Delete) of a VPS or SSD Cloud server hosted on Linode.

Where to start?

To implement a VPS CRUD you need to have a Linode account and obtain a API key from profile API keys section and use this obtained API key in Linode API client gem to create API client.

Once you install this gem, instantiate a API client as

API_KEY = 'xyz'
@client = Linode.new(api_key: API_KEY)

Before you proceed there are some prerequisites you need to know and have them configured

1. One or more template/source Linodes, which will be used while spawning/creating instances by cloning
  • Source Linode is a instance with all your necessary setup.
  • This source Linode has running Rails application or scripts that can be executed remotely or services you need to trigger remotely.
  • Once you launch source linodes and deploy tested code on these instances next thing you need to do is ‘Power Off’ these source linodes.
  • This is very important to note that if you want to clone a new Linode from source Linode then source Linode instance needs to be in ‘Powered off’ state.
  • Also there is a limit of 5 active clone operations per source/template Linode.
  • I would recommend to have one source Linode per region/datacenter if you need more that one Linode and keep all in one group something like ‘Source Linodes’.
  • Once you have list of source linodes, configure list of source Linode ids with location as following which will be used in clone operation.
#List of powered off instances to be cloned from
SOURCE_LINODES = {
  london: 5679,
  newark: 1234
}
2. Configure List of Data Centers You Needed
  • You need to give a data center id while cloning a instance .
  • Essentially this is the location where your new instance will be spawned/created.
  • Linode offers 8 data centers.
#List of available data centers and data center ids
DATACENTERS = {
  newark: 6,
  london: 7
}
3. Configure Linode Plan Id
  • You need to give a plan id while cloning a instance.
  • Linode offers various plans for instances.
  • Configure the best plan as per your needs.
#Linode plan id used while launching new instances
PLAN_ID = 1

Once you configure above all then we are set to do a CRUD of instances.

Let’s do CRUD of Instances

Creating and Booting an Instance

Launching and Booting  Linode using source linodeid, datacenterid and planid.

#Launching instance
@linode = @client.linode.clone(
  linodeid: SOURCE_LINODES[:london],
  datacenterid: DATACENTERS[:london],
  planid: PLAN_ID
)
#Booting instance
@linode.boot(linodeid: @linode.linodeid)
Updating an Instance

Updating Linode label, display group by linodeid

#Updating instance
@client.linode.update(
  linodeid: @linode.linodeid,
  label: 'Instance' + '-' + @linode.linodeid.to_s,
  lpm_displaygroup: 'My Linodes'
)
Listing all Instances
#Listing instances
@client.linode.list
#[#<OpenStruct datacenterid=6, lpm_displaygroup='My Linodes', totalxfer=600, alert_bwquota_enabled=1,...]
Getting IP of Instance
data = @client.linode.ip.list(linodeid: @linode.linodeid)
#[#<OpenStruct ipaddress='209.123.234.161', linodeid=12446, ...]
data.first.ipaddress
#209.123.234.161
Deleting Instance

Deleting Linode by linodeid
Note: ‘skipChecks’ option which Skips the safety checks and will always delete the Linode
its kind of force delete.

@linode.delete(linodeid: linodeid, skipChecks: true)

Ideal Scenarios where We Need Spot Instances

  • First and more common use of the spot instances is for web scraping by having farm of instances.
  • If you need to execute various services on different servers which takes long time to complete.
  • If you have limited infrastructures for your services and you want to replicate your service on other server very quickly.

I have created a template gist which is a consolidation of all the steps that I explained above.

If you got stuck in some problem while implementing this, Please let me know It will be my pleasure to help you out and suggestions are most welcome.

Have a nice day!

Advertisements

Are your workers working?

Most of the Rails developers choose Sidekiq for background processing in their Rails applications and setup the Sidekiq workers to do the long running work in background.

But as a Rails developer have you ever wondered whether your workers are doing their job? Now question may pop up in your mind, How to know whether my background workers doing their job? No worries now you can do that easily with sidekiq-status gem, How? Lets see how …

Bit about sidekiq-status:

  • An extension to the sidekiq message processing to track your jobs.
  • sidekiq-status implemented sidekiq middleware and uses Redis underneath for storing statuses and meta information about jobs.
  • In latest release gem added some really cool features like more status types, Web UI enhancements  which makes it really cool to have in your projects for tracking background jobs.

Let’s setup sample worker:

To track job progress of worker add sidekiq-status gem to the Gemfile and after `bundle install` include `Sidekiq::Status::Worker` module in your worker.

Lets say one wants send  emails to 100 different users in background Sample `UserWroker` will look like,

  class UserWorker
    include Sidekeq::Worker
    include Sidekiq::Status::Worker #Important
    def perform
      User.find_each do |user|
        UserMailer.weekly_summary(user).deliver_now
      end
    end
  end

Now make your worker more proactive by using following methods from `Sidekiq::Status::Worker` module to set the current progress of the worker.

  • total : Total number of actions that your worker going to perform default is 100 eg. If your worker sends email to 100 users set this to 100.
    total 100
  
  • at : Sets current number of actions performed by worker eg. If job has completed sending email to 10 users then worker sets this value to 10.
    at 10, "some message"
  
  • retrieve : Method to retrieve values by method name or stored keys eg. If you want to retrieve value of ‘at’ then `retrieve :at` will return latest value of ‘at’.
    value = retrieve :at #returns 10
  
  • store : Associate any custom data with job, eg. If you want to store delivery count in status hash.
    store delivery_count: 10
    delivery_count = retrieve :delivery_count #returns 10
  

Let’s use above methods to track job status in our sample worker which will turn into following

  class UserWorker
    include Sidekeq::Worker
    include Sidekiq::Status::Worker #Important
    def perform
      # Setting total number of users
      total User.count
      store delivery_count: 0 
      User.find_each do |user|
        # Setting 'at' for progress
        status = retrieve(:at).to_i
        delivery_count = retrieve(:delivery_count).to_i
        at (status.to_i + 1), "User: #{user.id}, delivery_count: #{delivery_count}"
        UserMailer.weekly_summary(user).deliver_now
        delivery_count += 1
        store delivery_count: delivery_count
      end
    end
  end

After setting your worker like above, visit /sidekiq/statuses tab on sidekiq web you should see your workers progress.

sidekiq-status-blog

On cautious note, make sure you follow all steps to integrate Sidekiq  Web UI and sidekiq-status Web UI.

I have created a template gist for above sample worker along with detailed steps to setup sidekiq worker with sidekiq-status.

If you got stuck in some problem while implementing this, Please let me know It will be my pleasure to help you out and suggestions are most welcome.

Have a nice day and keep watch on your Workers!