Apache Kafka – Introduction

Apache Kafka is the distributed messaging system which serves as a substitute for traditional JMS messaging systems in the world of BIG-DATA. Another way to describe Kafka as per Apache website is “Apache Kafka is publish-subscribe messaging rethought as a distributed commit log”. It was originally developed by LinkedIn and later on became a part of the Apache project.

Features of Kafka are
  • Faster
  • Scalable
  • Durable and
  • Distributed by Design.

Kafka has some differences when compared to other message brokers like RabittMQ, Websphere MQ . One of the best-known advantages is its performance. It consumes data in its own way.

Components of Apache Kafka

There are five important components in Kafka as given below

Kafka COMPONENTS

Kafka can have multiple producers and consumers and work as a cluster in a distributed model.

Kafka works on a publisher-consumer mechanism. Kafka maintains feeds of messages in  “topics“, processes which publish messages to a Kafka topic is named as “Producers“, processes that subscribe to the topic and process the published messages is called as “Consumers“. Kafka is run as a cluster comprised of one or more servers each of which is called as a broker.

Kafka Use cases
  • Messaging: Replacement for a more traditional message broker, Kafka has better throughput, built-in partitioning, replication, and fault-tolerance.
  • Website Activity Tracking: Real-time processing, real-time monitoring, and loading into Hadoop or offline data warehousing systems for offline processing and reporting.
  • Metrics: Aggregating statistics from distributed applications to produce centralized feeds of operational data.
  • Log Aggregation: Collects physical log files of servers and puts them in a central place (a file server or HDFS perhaps) for processing

This is just an introduction about Kafka, please refer Apache-site documentation for detailed documentation.

Posted on

How to Transfer/Refer Jmeter Variables from one Thread Group to another Thread Group?

In many situations, we encounter in referring the variables from one thread group to another. Even I found myself every time I end up in searching for the solution. Finally, I have decided to write it here, there are many ways to transfer variables, I preferably choose the following method.

1) Set up Thread Groups
2) Add a “BeanShell Assertion” to the sampler where you are extracting the value
3) In the Script area of BeanShell Assertion specify as
${__setProperty(SG_Id2,${SG_Id1})} where SG_Id2 is a variable in which we are storing the value of
SG_Id1 , so that we can refer this value anywhere in the testplan across all the thread groups.
4) Now go to next Thread Group sampler where you want to pass the value and give as
${__property(SG_Id2)}
5) Now run the test and check the value passed from one thread group to another thread group

Posted on

Introduction to Elasticsearch

Elasticsearch is an Open Source  (Apache 2), Distributed search engine built on top of Apache Lucene. It allows you start with one machine and scale to ‘n’ number of servers with high availability.

Elasticsearch makes it easy to run a full-featured search server. It can be setup in lesser than five minutes. In this blog I’ll show how to:-

  • Install and run elasticsearch.
  • Indexing data.
  • searching
Installing and running Elasticsearch
  1. Download and unzip the latest version of Elasticsearch from elastic.co website.
  2. Go to the extracted folder, and Run bin/elasticsearch  on Unix or bin/elasticsearch.bat  on Windows, your terminal will be showing something like below.

elastic_9200_terminal

3.   Open web browser and point the url to http://localhost:9200/ , you should see something like below

elastic_9200

Indexing data

There are to main ways in adding data to elasticsearch.

  1. json over HTTP
  2. Native client.

Here we are using the curl command to insert data into elasticsearch

$ curl -XPUT ‘http://localhost:9200/twitter/tweet/1’ -d ‘{

“user” : “kimchy”,

“post_date” : “2009-11-15T14:12:12”,

“message” : “trying out Elasticsearch”

}’

the result of the above index operation is:

{

“_index” : “twitter”,

“_type” : “tweet”,

“_id” : “1”,

“_version” : 1,

“created” : true

}

Posted on

Illustration of MITM attack

MITM-diag
MITM illustrated

Posted on

Gif: Illustration of Manual transmission.

Here’s a gif, from internet illustrating manual gear transmission.

How_Manual_transmission_works

Posted on

Installing .NET Framework 3.5 on Windows Server 2012 and Windows Server 2012 R2

There might be requirements of installing .Net Framework 3.5 in Windows Server 2012 and Windows Server 2012 R2, and you will most likely run into problems while installing it.

If you are trying to install .NET Framework 3.5 from the Server Manager GUI, you will see this when installing the feature:

“Do you want to specify an alternate source path? One or more installation selections are missing source files…”

Confirm

 

 

To solve this, you can either:

1. Go to a command prompt and enter this:

dism /online /enable-feature /featurename:NetFX3 /all /Source:d:sourcessxs /LimitAccess

Note: Source should be the Windows installation disc. In my case, this was located on D:

command

2. Go down to “Specify an alternate source path” and enter “d:sourcessxs” as the path.

addroles

addroles

Now you can proceed with installation.

Posted on

How to execute a test continuously for specified duration without bothering no. of times using JMETER?

Scenario:

Do you have the following scenario to execute by Jmeter and not able to find the solution?

Having HTTP Request / REST Request in a test plan, required to execute the test for 1 hour duration without bothering repetition , and to

  • Find the application robustness and consistency to bear max load?
  • Find no. of requests application can handle for the specified duration?

and you would have

  • tried giving in scheduler but not worked as it executes for the number of threads specified In thread group.
  • tried giving in Duration but not worked. So what is next?

Solution:

  • Check ‘Forever’ check box in Thread group
  • Give Duration as ‘3600’ in Scheduler section of Thread Group

image

Enjoy Testing  Smile

Posted on

MySQL an introduction and basic tutorial

MySQL is a Open source database management systems. It ‘s a powerful database management system with a lot of flexibility. This tutorial covers basic introduction to MySQL. In this tutorial I’m using Centos operating system for installing MySQL.

Installing MySQL in centos

MySQL can be installed using yum repository, and can be installed using the below comment.

#yum install mysql-server

image

Once the installation is complete, you can start mysql using the below comment.

#/etc/init.d/mysqld start

No password is set by default during MySQL installation.

For setting up password for MySQL for the first time use mysqladmin to set root password

mysqladmin -u root password NEWPASSWORD

eg: mysqladmin –u root password Password123

Accessing MySQL Shell

MySQL prompt can be accessed using the below command

#mysql –u root –p

and enter the MySQL password created.

image

Now we are in MySQL prompt.

How to list databases?

type ‘show databases’ at mysql prompt.

image

How to create a database?

Syntax: create database <dbname>

image

How to access a database?

Syntax: use <dbname>

image

How to delete a database?

Syntax: drop database <dbname>

image

Posted on

Apache JMeter HTTP(S) Test Script Recorder

In this tutorial we are planning to explain step by step on how to record HTTP/HTTPS sessions.

Setup Instructions

  1. Go to JMETER_HOME/bin and start JMETER with jmeterw.cmd and jmeter in linux/unix.
  2. Select “Test Plan”, right click on “Test Plan” and add a new thread group: Add > Threads  (Users) > Thread Group.

image

3.  Select “Thread Group” right click, “Add –> Config Element –> HTTP Request Defaults”

image

4.  In new HTTP Request Defaults element: Server name – enter “host to be tested”, path can be left blank.

image

5. Right click on the “Thread Group” and add a recording controller: Add > Logic Controller >Recording Controller.

image

6.  Next, select WorkBench, Right click on WorkBench and add the recorder: Add -> Non-Test Elements -> HTTP(S) Test Script Recorder

image

7. On HTTP(S) Test Script Recorder, click the “Add” button in “URL Patterns to Include”. This will create a blank entry; enter “.*.html”. you can specify URL patterns to include and exclude such as *.html, *.js, which you would like to record.

image

8.  Right click on “HTTP(S) Test Script Recorder” and add a listener: Add -> Listener –> View Results Tree

image

9. Return to HTTP(S) Test Script Recorder, and click the “Start” button at the bottom.

This will start the JMeter proxy server which is used to intercept the browser requests. A file called ApacheJMeterTemporaryRootCA.crt will be generated in jmeter/bin folder. Install this certificate in your browser.

Note: If you browser already uses a proxy, then you need to configure JMeter to use that proxy before starting JMeter, using the command-line options -H and -P.

Configure Firefox To Use JMeter Proxy

We will use Firefox as our browser when using the JMeter HTTP(S) Test Script Recorder because, unlike Chrome and some other browsers, it does allows you to override system-wide configuration for its proxy settings.

Configure Firefox to use localhost (127.0.0.1) on port 8080 as its proxy for all traffic by following these steps:

  1. Open Firefox
  2. Go to the Preferences menu
  3. Click on the Advanced tab
  4. Then Network tab
  5. In the “Connection” section, click on “Settings…”
  6. Select the “Manual proxy configuration” radio button
  7. Set HTTP Proxy to “localhost” and Port to “8080”
  8. Check “Use this proxy server for all protocols”
  9. Click OK and exit the Preferences menu

Note: When Firefox is configured to use JMeter’s Script Recorder as a proxy, it will only work properly if the Script Recorder is running.

Recording HTTP Requests

Now that our test plan’s HTTP(S) Test Script Recorder is running, and Firefox is configured to use it as a proxy, the HTTP requests that Firefox sends will be recorded. Let’s test it out.

In Firefox, go to your server’s homepage (the same server that you configured in your JMeter HTTP Request Defaults):

http://your_domain.com/

Now there should be a little triangle next to your Recording Controller. Click on it to expand and show the requests that it has recorded. You should see the HTTP requests that were recorded, depending on which URL Patterns you have included and excluded. Feel free to browse your site to record more requests.

As you can see, a lot of requests were created. You may refine the list of HTTP requests by simply deleting unwanted entries here.

If you do not see any entries under your Recording Controller, you will want to review your URL Patterns in the HTTP(S) Test Script Recorder (Hint: Remove all includes and excludes to record everything).

Once you are done recording, click the “Stop” button at the bottom of the HTTP(S) Test Script Recorder window. Note that Firefox will no longer be able to reach any pages (because it is configured to use port 8080 as a proxy)–configure it to use “No proxy” if you want to function normally.

Run Your Test Plan

Once you are happy with the test plan you have recorded, save it, then run it. It will function exactly like a manually created test, so you can configure it, delete, and add items to make it match your desired test case more closely.

Posted on

Microsoft Antimalware for Azure services available for free

The solution is built on the same antimalware platform as Microsoft Security Essentials [MSE], MicrosoftForefront Endpoint Protection, Microsoft System Center Endpoint Protection and Windows Defender for Windows 8.0 and higher. Microsoft Antimalware for Azure is a single-agent solution for applications and tenant environments, designed to run in the background without human intervention. You can deploy protection based on the needs of your application workloads, with either basic secure-by-default or advanced custom configuration, including antimalware monitoring. When you deploy and enable Microsoft Antimalware for Azure for your applications, the following core
features are available:
Real-time protection – monitors activity in Cloud Services and on Virtual Machines to detect and block malware execution.
Scheduled scanning – periodically performs targeted scanning to detect malware, including actively running programs.
Malware remediation – automatically takes action on detected malware, such as deleting or quarantining malicious files and cleaning up malicious registry entries.
Signature updates – automatically installs the latest protection signatures (virus definitions) to ensure protection is up-to-date on a pre-determined frequency.
Antimalware Engine updates – automatically updates the Microsoft Antimalware engine.
Antimalware Platform updates – automatically updates the Microsoft Antimalware platform.
Active protection – reports telemetry metadata about detected threats and suspicious resources to Microsoft Azure to ensure rapid response to the evolving threat landscape, as well as enabling real-time synchronous signature delivery through the Microsoft Active Protection System (MAPS).

Samples reporting – provides and reports samples to the Microsoft Antimalware service to help refine the service and enable troubleshooting.
Exclusions – allows application and service administrators to configure certain files, processes, and drives to exclude them from protection and scanning for performance and/or other reasons.
Antimalware event collection – records the antimalware service health, suspicious activities, and remediation actions taken in the operating system event log and collects them into the customer’s Azure Storage account.

Microsoft Antimalware in Azure workflow – enable, configure, and monitor

image


How to enable and configure Microsoft Antimalware for Azure virtual machines?

There are multiple ways to do this, in this section I’m will let you know how to enable it using Azure management portal while provisioning a virtual machine.

  • logon to Azure management portal at https://manage.windowsazure.com
  • To create a new virtual machine, click New, Compute, Virtual Machine, From Gallery, as shown below.

image

  • Select the Microsoft Windows Server image on the choose an image page.
  • Click the right arrow and input the Virtual Machine configuration.
  • Check the Microsoft Antimalware checkbox under Security Extensions on the Virtual Machine configuration page.
  • Click the Submit button to enable and configure Microsoft Antimalware for Azure Virtual Machines with the default configuration settings.

image

Posted on