Author: Dasun Hegoda

hadoop 2 deployment on ubuntu

Hadoop 2 Deployment (Installation) on Ubuntu

Hadoop is everywhere and gaining attention like crazy. This is not an article which explains what’s it or how it works because there are a lot of good resources for that. So I don’t want to repeat the same stuff but I’m going to help you to go step further and deploy a Hadoop multi node cluster on ubuntu. Pretty interesting right? If you follow the steps given below you can get it done in 15 mins. Let’s start.

Prerequisites

All you need is

  • Java 1.7 should be installed.
  • 5 Nodes. In my case it’s 192.168.7.87, 192.168.7.88, 192.168.7.89, 192.168.7.90, 192.168.7.91

1. Configure Environment

  1. Let’s create a dedicated user for hadoop who’s hduser.
  2. Configure password-less SSH
    1st you will have to decide which node is going to be the master, the secondary master and the slaves. Then make sure that the master node is able to do a password-less ssh to all the slaves and the secondary master. If you don’t know how to setup password-less ssh refer this article.
  3. Edit /etc/hosts and add the below. Also comment out IPV6.
  4.  Edit hostname file
    In the master node edit the hostname file as shown below.

    just replace the content with master. Now follow the same steps and edit the hostname in other nodes as well. The hostname should be master2, slave1, slave2, slave3 respectively.

– Read More –

REST API Architecture

REST API Architecture – Best Practices

SOA(Service Oriented Architecture) has become a foundation for the most of the application that are developed today. A service oriented architecture is an architectural pattern which enables collection of services to communicate with external/internal parties to pass data or for services coordinating. Let’s start from what’s a service.

What’s a Service/API

A service is a function that is well-defined, self-contained, and does not depend on the context or state of other services.

In other words an interface used by software components to communicate with each other. Let understand the difference between API and Web Services. A Web Service is a type of API, almost always one that operates over HTTP. In the modern world there are two types of web services that are used.

  • SOAP
  • REST

We can’t compare REST and SOAP since SOAP is a protocol and REST is an architectural pattern. People mostly get confused when selecting one for their application. If you are wondering what’s the difference between these two you can get a good start from here and here.

Our today’s focus is on REST. How we can implement a perfect REST API? What are the best practices? Let’s get down to business.
– Read More –

Linux Command

Top 10 Linux Command Examples

Today let’s focus on a Linux command list that will be useful to a newbie who is just starting off with Linux. It doesn’t matter whether you are a web developer or software engineer if you are going to use Linux you have to learn these. These commands are used day to day to get done work.

I don’t want to talk about the typical command list that is on the internet rather I would like to go beyond that and introduce couple of useful Linux commands. For the below commands you can find tons of tutorials.

  • cd
  • mv
  • cp
  • ls
  • pwd
  • ifconfig,
  • mkdir

So let’s move on.
– Read More –

High Availability Deployment Architecture

What’s High Availability

High availability is a critical success factor for any given enterprise application. Today we’ll go through how to design and deploy an application with high availability. First things first. Let’s see what’s high availability.

High availability is a characteristic of a system, which describes the duration (length of time) for which the system is operational. – Wikipedia

In simple words high availability can be defined as running a system 24*7 without a downtime even if there are hardware and software failures. In other way a fault tolerance application. This helps ensure uninterrupted use of the application for it’s intended users. If you need more information you can read this article. Also you might be thinking when is the right time to introduce high availability in your application? The answer is here.

Following is an architecture that supports high availability. It’s the minimal requirement to implement high availability in your application(Note that in the database cluster you can have master-slave instead of master-master depending on your requirement).

High Availablity Deployment Architecture

High Availablity Deployment Architecture

– Read More –

SSH Tunnelling – Local Port Forwarding

Introduction

Today we are going to experience the power of the SSH(Secure Shell) command. Yes Linux. I recently wrote an article on X11 forwarding that allows to map your local computer’s GUI to server where SSH is used underneath. It’s time for you read it if you haven’t. To understand the today’s topic let’s 1st look at what’s tunnelling that’s the foundation. In wikipedia tunnelling is defined as follows.

In computer networks, a tunneling protocol allows a network user to access or provide a network service that the underlying network does not support or provide directly. – wikipedia

SSH tunnelling can be created using three kinds of port forwarding mechanisms as mentioned below.

  • Local port forwarding – Today’s topic
  • Remote port forwarding
  • Dynamic port forwarding

Local Port Forwarding via SSH

Today we are going to focus on local port forwarding. This can be used to access a network that is not accessible directly. Let’s take an example and see. The following image is used to elaborate the scenario.

Local Port Forwarding Using SSH

– Read More –

Older Posts
Newer Posts