Menu

Blog

Explore our insightful blog posts that delve deeper into the latest industry trends, best practices, and success stories. Gain valuable knowledge and stay informed about the ever-evolving landscape of digital transformation.

Python Automation: Streamlining Data Workflows

In today’s data-driven world, the ability to automate routine tasks is a game-changer. Python, with its extensive libraries and scripting capabilities, empowers professionals to form streamlined data workflows, save time, reduce errors, and improve efficiency. In this document, we’ll delve into the technical aspects of Python automation, exemplified by a use case.The Power of Python AutomationConsider a scenario where a data analyst is tasked with collecting data, performing analysis, and sending daily reports to stakeholders. The process, if done manually, is laborious and prone to errors. Python automation can revolutionize this workflow.Use Case: Data Analyst’s DilemmaMeet Alex, a data analyst grappling with a substantial daily workload. Gathering data from various sources, performing analysis, and distributing reports was a time-consuming, repetitive process. The need for timely and accurate reports often led to stress and inefficiency. Determined to optimize this workflow, Alex turned to Python automation.How Python Automation Transformed Alex’s WorkflowAlex took a systematic approach to address the data collection, analysis, and reporting aspects of the task:Import the required Lib -#importing required librariesimport pandas as pdimport numpy as npimport pytzimport gspreadimport pygsheetsimport datetimefrom datetime import timedeltafrom datetime import datetime#import oauth2clientfrom queryrunner_client import Client|from oauth2client.service_account import ServiceAccountCredentialsgs = pygsheets.authorize(service_file 'example. j son' )c Client(user_email 'example@abc.com' )STEP 1-Explanation as to what each does and why is it necessary -1. pandas (`import pandas as pd`):- Purpose: pandas is a powerful data manipulation library. It provides data structures like DataFrames for efficient data manipulation and analysis.- Why Required: Used to handle and manipulate tabular data, such as reading/writing data, merging, filtering, and performing various data operations.2. numpy (`import numpy as np`): - Purpose: numpy is a numerical computing library. It provides support for large, multi-dimensional arrays and matrices, along with mathematical functions to operate on these arrays.- Why Required: Used for numerical operations and working with arrays, especially in scenarios where efficient numerical computations are needed.3. pytz (`import pytz`):- Purpose: pytz is a library for working with time zones. It allows the localization of datetimes to different time zones.- Why Required: Used to handle and convert time zones, ensuring accurate representation and manipulation of datetime values.4. gspread (`import gspread`):- Purpose: gspread is a Python API for Google Sheets. It allows for interaction with Google Sheets, such as reading and writing data.- Why Required: Used to connect to and manipulate Google Sheets data, which might be the source or destination for data.5. pygsheets (`import pygsheets`):- Purpose: pygsheets is another library for working with Google Sheets, providing additional functionality and ease of use.- Why Required: Used for authorization and access to Google Sheets. Offers a convenient way to work with Google Sheets using Python.6. datetime (`import datetime`):- Purpose: datetime is a module in the standard library for working with dates and times.- Why Required: Used for handling datetime objects, calculating time differences, and formatting timestamps.7. timedelta (`from datetime import timedelta`):- Purpose: timedelta is a class in the datetime module for representing differences between two dates or times.- Why Required: Used for adding or subtracting time intervals, such as calculating a date in the future or past.8. Client (`from queryrunner_client import Client`):- Purpose: queryrunner_client is a library for interacting with a query runner service, allowing the execution of queries and retrieval of results.- Why Required: Used for querying a data source or executing specific operations, depending on the service it connects to.9. ServiceAccountCredentials (`from oauth2client.service_account import ServiceAccountCredentials`):- Purpose: ServiceAccountCredentials is used for OAuth 2.0 authentication with Google services.- Why Required: Used to authenticate the application and authorize access to Google services, such as Google Sheets.10. pygsheets authorization (`gs = pygsheets.authorize(service_file=’gcp_creds_example.json’)`):- Purpose: Authorizes access to Google Sheets using credentials from a service account file.- Why Required: Enables the Python script to interact with Google Sheets on behalf of the service account.STEP 2-Now give the Gsheet that you want to play around with or share it across to your managers for their view —Sheet = “Link of the sheet which is shared to your managers”Then we authorise and open the sheet -If you are looking to use this sheet in your Python code, you might want to extract the sheet key from the URL. The sheet key is the long alphanumeric string between “/d/” and “/edit” in the URL.For example, in the URL you provided:`https://docs.google.com/spreadsheets/exampleThe sheet key is `djskhuej6778d9e3nbsk`.If you intend to use this sheet in your Python code with the `gspread` library, you would typically use the key to open the sheet. Here’s an example:import gspread# Extract the sheet key from the URLsheet_url = ‘https://docs.google.com/spreadsheets/examplesheet_key = sheet_url.split(“/d/”)[1].split(“/”)[0]# Authorize and open the sheetgc = gspread.authorize(credentials)worksheet = gc.open_by_key(sheet_key).sheet1Make sure to replace `credentials` with your actual Google Sheets credentials. If you have a service account JSON file, you can use `gspread.service_account(filename=’path/to/credentials.json’)` for authorization.STEP 3-Now we write a function which is designed to update a specific range of cells in a Google Sheet with the data provided (which is basically the task you’re supposed to do. It can be scraping the data, running your queries and updating the sheet with that data or anything for that matter). Let’s break down each part of the function:def update_cells(sheet_link, tab_name, start_cell, end_cell, data):book = pygsheets_client.open_by_url(sheet_link)ws = book.worksheet_by_title(tab_name)ws.clear(start=start_cell, end=end_cell)ws.set_dataframe(df=data, start=start_cell)1. sheet_link`:- This parameter is the URL of the Google Sheet.2. `tab_name`:- This parameter is the name of the specific worksheet (tab) within the Google Sheet that you want to update.3. `start_cell` and `end_cell`:- These parameters specify the range of cells to clear and update in the worksheet.4. `data`:- This parameter is the DataFrame (`df`) containing the data you want to update in the specified range.5. `book = pygsheets_client.open_by_url(sheet_link)`:- Opens the Google Sheet using the provided URL.6. `ws = book.worksheet_by_title(tab_name)`:- Gets the specific worksheet (tab) within the Google Sheet by its name.7. `ws.clear(start=start_cell, end=end_cell)`:- Clears the specified range of cells in the worksheet to ensure a clean update.8. `ws.set_dataframe(df=data, start=start_cell)`:- Sets the provided DataFrame (`data`) starting from the specified cell. This updates the content in the specified range.Note: Make sure that the `pygsheets_client` variable is defined and contains the necessary authorization information to access the Google Sheet.Analyze your data, generate the necessary numbers and reports in your code. Once ready, invoke the `def_update_cells()` function, providing the required inputs. Presto! Your Google Sheet is seamlessly updated using Python.STEP 4-But how do I run my Python Script everyday at 5PM well there is a solution for that too —### Method 1: Using `cron` on Unix/Linux/Mac1. Open the crontab configuration:```bashcrontab -e```2. Add a new entry to run your script daily at 5 PM:```bash0 17 * * * /path/to/python /path/to/your_script.py```Replace `/path/to/python` with the path to your Python interpreter and `/path/to/your_script.py` with the full path to your Python script.3. Save and exit the crontab editor.### Method 2: Using Task Scheduler on Windows1. Open Task Scheduler:- Press `Win + R`, type `taskschd.msc`, and press Enter.2. Create a Basic Task:- Click on “Create Basic Task…” on the right-hand side.3. Follow the Wizard:- Name your task and provide a description.- Choose “Daily” and click Next.4. Set the Daily Trigger:- Specify the start date and time (5:00 PM) and set it to recur every 1 day. Click Next.5. Choose Action:- Select “Start a Program” and click Next.6. Configure the Program:- Browse and select the Python executable (`python.exe`) in the “Program/script” field.- In the “Add arguments (optional)” field, provide the full path to your Python script (`/path/to/your_script.py`).- Click Next.7. Review and Finish:- Review your settings and click Finish.Now, your Python script will be scheduled to run daily at 5 PM.Choose the method that corresponds to your operating system, and adjust the paths and file names accordingly.STEP 5-Now how do we let our end users know that sheet is updated with most recent data again Python is here to help you -Here are two different ways to set up mail sending using Python: one using the built-in `smtplib` library for sending emails through a Gmail account and another using a third-party library called `yagmail`.### Method 1: Using `smtplib` (with Gmail)Step 1: Enable “Less Secure App Access” in your Gmail account- Go to your Google Account settings: [Google Account](https://myaccount.google.com/).- Navigate to the “Security” tab.- Under “Less secure app access,” turn on access.Step 2: Write Python Code using `smtplib`import smtplibfrom email.mime.text import MIMETextfrom email.mime.multipart import MIMEMultipartdef send_email(subject, body, to_email):# Your Gmail credentialssender_email = ‘your_email@gmail.com’sender_password = ‘your_password’# Create the email messagemsg = MIMEMultipart()msg[‘From’] = sender_emailmsg[‘To’] = to_emailmsg[‘Subject’] = subject# Attach the body of the emailmsg.attach(MIMEText(body, ‘plain’))# Connect to the Gmail serverwith smtplib.SMTP(‘smtp.gmail.com’, 587) as server:server.starttls()server.login(sender_email, sender_password)# Send the emailserver.sendmail(sender_email, to_email, msg.as_string())# Example usagesend_email(‘Test Subject’, ‘Hello, this is a test email.’, ‘recipient_email@example.com’)```Replace `’your_email@gmail.com’` and `’your_password’` with your Gmail credentials. Ensure you keep your credentials secure.### Method 2: Using `yagmail` LibraryStep 1: Install `yagmail`**```bashpip install yagmail```Step 2: Write Python Code using `yagmail`**```pythonimport yagmaildef send_email(subject, body, to_email):# Your Gmail credentialssender_email = ‘your_email@gmail.com’sender_password = ‘your_password’# Connect to the Gmail server using yagmailwith yagmail.SMTP(sender_email, sender_password) as yag:# Send the emailyag.send(to=to_email, subject=subject, contents=body)# Example usagesend_email(‘Test Subject’, ‘Hello, this is a test email.’, ‘recipient_email@example.com’)```Replace `’your_email@gmail.com’` and `’your_password’` with your Gmail credentials. Ensure you keep your credentials secure.These examples assume you’re using a Gmail account. If you’re using a different email provider, you may need to adjust the SMTP server and port accordingly. Always be cautious with your email credentials and consider using environment variables or a configuration file for a more secure setup.Author: Abhishek Singh

Learn More >

The Journey from System Admin to DevOps Superstar

According to a report by the Economic Times, when organizations cultivate a better work environment, the overall experience improves exponentially. They find true meaning in their jobs by prioritizing employees’ mettle, exceeding expectations, and work allocation.Employees seek exposure and opportunities in their jobs. By building productivity and customer satisfaction they enhance their portfolio.Radhakrishnan one of our DevOps superstars, has contributed with his service and time for over 8 years. To commemorate this everlasting relationship we got into a candid conversation with him. Here’s what he had to say about his journey before and with Nineleaps.Radhakrishnan is originally from a small town near Bengaluru, Hosur. After completing his MBA, his interest developed in computers and networking. He successfully gained appropriate knowledge by undertaking network courses and embarked on a journey to becoming a system admin. He enjoyed working for various companies as a system admin.Then came Nineleaps which gave new horizons of opportunities to his mettle. When we asked him about his transition from a system admin to a DevOps engineer, he fondly remembered a quote given to him by our CEO on the day of his selection.“You are on the flight now, just fly,” — Divy Shrivastava.And, so he did.Divy’s words of confidence boosted his resolve. The walk towards DevOps became a sprint, as multiple iterations of knowledge and experience suffused him. The arena of his work leaped and much to his admiration, he realized DevOps to be his passion and soul.Right from the get-go, an intensive training regimen, honing his skills, immersing himself in countless hours of study, and shadowing esteemed senior members of our organization he grasped the crucial importance of comprehending tasks and prioritizing them effectively. Driven by an unwavering desire to learn and prove his mettle, his transition from a system admin to a DevOps maestro was seamless. Multiple training sessions helped him get a deeper understanding of internal and external projects as well as the product, giving him never-to-dull confidence.Learning and development, knowledge transfers, and peer learning are certainly at the core of Nineleaps which helped him become the super engineer he is today. These trainings were both from the client’s side as well as in-house learning at Nineleaps.“In my opinion what sets Nineleaps apart is our dynamic and flexible approach to projects, with extensive focus on Agile methodology we are trained and nourished to build quality solutions for our clients, and also are facilitated with high-tech exposure by working with industry giants and rewarded with the utmost respect and growth opportunities.”To understand more closely we asked him about the challenges he faced at times, and according to him, documentation was a challenge. He feels all the work that the employee is doing must be documented and organized in a proper way as it will help them in the future. He also informed about instances where a person working on a specific problem might face similar challenges later in the same week and not be able to recall what the solution was properly, in such cases documenting everything became important. The organization’s culture was very open and asking questions or requesting help was never an issue which facilitated collaboration in resolving such challenges.Nineleaps became the crucible to test his mettle and with each strike of the hammer, a superstar was born.

Learn More >

Becoming the Right Mentor for Emerging Female Talent

In today’s rapidly evolving professional landscape, achieving gender equality in the workplace remains a crucial goal. Women’s Equality Day serves as a reminder of the progress we’ve made and the work that still lies ahead. One key aspect of this progress is nurturing and mentoring emerging female talent, which was the focal point of a recent panel discussion. The discussion revolved around the theme “Becoming the Right Mentor for Emerging Female Talent” and brought to light several key takeaways that highlight the importance of mentorship, teamwork, and personal growth in the pursuit of gender equality and professional excellence.A Positive Work Environment and Collaborative TeamworkA resounding theme from the panel discussion was the significance of a positive work environment and collaborative teamwork in fostering professional growth and satisfaction. Effective leadership places a strong emphasis on team dynamics, recognizing that a harmonious work environment not only boosts productivity but also supports individual development. A culture of collaboration is pivotal in promoting diversity, ensuring that all voices are heard, and driving organizational excellence.Competence Over BiasesEquality based on competence, rather than conforming to biases, emerged as a fundamental principle for success and diversity in the workplace. The panel highlighted that recognizing and appreciating the unique skills and abilities of each individual, regardless of gender, is crucial. It is only by breaking free from preconceived notions and biases that true progress can be made.Continuous Learning and AdaptabilityThe importance of continuous learning and adaptability was a recurring theme throughout the discussion. In today’s ever-changing world, the ability to learn and adapt is essential for personal development and overcoming challenges. Panelists emphasized the need for women to embrace a commitment to lifelong learning as it fuels both personal and professional growth throughout their careers.Overcoming Self-DoubtSelf-doubt is a common hurdle faced by many, particularly women, in their professional journeys. The panelists encouraged women to believe in their potential and not let self-doubt hinder their capabilities. Confidence in one’s abilities is a cornerstone of success and empowerment.Transformative Impact of MentorshipThe impact of mentorship, both traditional and reverse mentoring, on employees was highlighted as transformative. Mentorship provides guidance, support, and a platform for individuals to learn and grow. It was emphasized that mentorship should not be defined by gender but rather by skills, with a focus on finding the right mentor-mentee match.Proactive Communication and TrustProactive communication and trust within a team were identified as key factors in achieving clear expectations and productive outcomes. Trust, especially during challenging times, enhances collaboration and problem-solving capabilities. Open and honest communication fosters an environment where everyone feels valued and heard.Lifelong Commitment to LearningThe commitment to lifelong learning was underscored as a driving force behind personal and professional growth. Panelists stressed that embracing new opportunities and continuously seeking knowledge are essential aspects of staying relevant and thriving in the modern workplace.Meaningful MentorshipFinally, meaningful mentorship, grounded in understanding individual strengths and weaknesses, was recognized as integral to fostering individual development and valuable work contributions. It is through this personalized guidance and support that emerging female talent can reach their full potential.The key takeaways from the Women’s Equality Day panel discussion on “Becoming the Right Mentor for Emerging Female Talent” reinforce the notion that fostering gender equality and professional excellence requires a holistic approach. Creating positive work environments, valuing competence over stereotypes, and committing to continuous learning are essential steps. Moreover, mentorship, self-belief, and effective communication play pivotal roles in nurturing emerging female talent and ensuring a brighter, more equitable future in the workplace.

Learn More >

A Diverse Programming Languages Guide

People who have been following me kind of know that I have been mostly writing about machine learning these days. But it’s the end of the year and everyone is coming up with their best programming languages for year 2018 and what could be the programming languages for the year 2018. So, I also thought that I should also say something on the topic and not let the chance go by. This post is for those developers who are seasoned in Python and are looking for ways to improve their skills and become more attractive in the job market. In case you are a beginner still this post may have something for you. As a bonus I will tell you the programming language you need to start with. Also, please note that the listing here does not mean that I am ordering them. Please take into account your individual goals and motivations then choose the language that you would like to learn.JavaScript/ReactThese days there is a lot of good things that are going good for JavaScript. In case you have been living under the rock for the past 1000 years, everything is mobile. Internet speeds are only going to increase and internet has become a basic need for a huge portion of the population. If you have been ignoring and avoiding JavaScript, now is the best time to learn it. The new ECMA6 has a lot of new and improved features which make learning and using the language easy. Then there are a lot of libraries that are there in JavaScript one of the most used for web programming is React. Though React had some backlash from the open source community due to its licensing, Facebook, who is maintaining the library changed the license to make it more developer friendly. So, keep calm and accept JavaScript.C#That was web programming. Although Python has many advantages but one of the key disadvantages is that it is notoriously difficult to create native apps. If that is the case, why should you as a developer disadvantage yourself. Take a look at Xamarin. You would write in C# and easily deploy in Android, iOS and even desktop apps. I would suggest learn basic C# and Xamarin, keep all your business logic in flask endpoints and easily create android and iOS apps on the fly with minimal code.RustArmin loves Rust and hence I love Rust. So, you should learn Rust. Period.The above statement should be your biggest motivation, but in case you are not convinced, there are many good things going on with Rust. The fact that it does not have a runtime makes it an excellent candidate for those small patches of code that take a lot of memory and CPU. In case you are looking to make code optimizations and were thinking about C or C++, take a look at rust. It might be just right for you and your team. Take a look at this excellent article on the same.C/C++Although I don’t really like these languages, the fact of the matter is there is a lot of legacy code in C and C++. Also Python is written in C and if you want to get into the internals of Python you should probably know C. Another way to look at it is that the current trends are Machine Learning, Artificial Intelligence and probably AR/VR. All of them are computationally intensive and hence, to wring out the most from your machines you probably will need to know some C and C++. Maybe this is the right time to get into those languages now that you have solid Python under your belt.PHPAll of the above are good choices but probably you are just interested in earning money. Or probably things are not going good at work and hence you are thinking of taking the freelancing route. In that case PHP should be a good bet. You may think why am I saying this although you may have heard a lot of people dissing it. Generally, tools are not inherently bad, its the programmers who are bad. An excellent example is that Facebook was able to build a massive empire based on PHP. Also, major portion of the web is based on WordPress and there is still a lot of wordpress related freelancing jobs. In fact, I would argue that its a good thing that people are speaking bad about WordPress. Think about it this way. Once the code in a lot of websites become old and are legacy, there will be high demand for people who know PHP and you will be able to demand a high charge for your services.ClojureNow lets get away from money and come to the world of the uber geeks, code quality and best code structure. If you fall in that realm, probably you should look at Clojure. You can choose any of the other functional programming languages as well but I will tell you why you should consider Clojure. IT WILL GIVE YOU A PURPOSE. Just imagine you started with Clojure and you have mastered the functional programming concepts and are writing pure code with microscopic amount of side effects.Interestingly, Clojure runs on JVM and you can now write Java apps. You now have the power to wage war against those corporate managers and team leaders and those corporate developers. They will write all those massive architectural designs for simple TODO apps while you will be doing the same thing in 50 lines of code. You will able able to piss off everyone. Just a little food for thought for the rebel in you.That’s it from me. In case you have some favorite lang which I have not included in this list do write them out in the comments. Also, I had promised that I will speak about the language that you should learn in case you are interested in programming and what to start in the field. The answer is easy. PYTHON.

Learn More >

Mastering Logistic Regression

Logistic Regression although named regression is actually a classification technique and not a regression. It is named regression because the technique is quite similar to linear regression which I had discussed in this post. Have a look at the post as it will help you in understanding the concepts. You will need to keep in handy the equations mentioned there. The term “logistic” is taken from the logit function that is used in this method of classification.In this post we will take a look at the math of how logistic regression works, code a custom implementation of the model and how to implement it using scikitlearn.What is Classification?Classification is termed when you want the output in buckets. For example, you have an email and you want to classify it as Spam and Not Spam. Or you have a lot of pictures of dogs and cats and you want a model that can classify the pictures into dogs and cats and appropriately label them. In more serious science maybe you have scans of tumors and you want to identify them as malignant or benign. All of these examples are classification problems and logistic regression can be employed for them.Problem with using regression in classificationNow you may say. “OK, but why should I take your word for it. I already know one machine learning technique which is linear regression. I will use that and be done with it.” To answer that question let’s try to use the principal of negation. Say, we are trying to fit a straight line through a sample dataset.In the above example malignant tumors get 1 and non malignant tumors get 0 and we are trying to fit the green line. While making predictions we will say that if the value on the line lies above 0.5 on the y axis (malignant?), then the tumor is malignant, else we will say it’s benign. We are happy with our predictions and we go home.But wait, the doctor (our customer) comes back to us and shows us the following problem in the model. He ran the model on his dataset and the model got trained a bit different and it’s giving erroneous results. You look at the dataset and the resultant green line and you see this.This is valid because all tumors that are large are also malignant. Now our model that malignant if y>0.5 does not work.We cannot change the hypothesis every time a new dataset comes. That would defeat the whole purpose. In technical terms, our model does not generalize. We have to find a better way of defining our model. Fortunately, we have Logistic Regression, which is Regression analysis but with a twist.Logistic Regression ModelIn the logistic regression model, we want our classifier to output values that are between 0 and 1. So we are going to come up with a hypothesis that has this limiting property.Linear regression gives us the below form to be used for fitting our model.For logistic regression the above equation is modified a bit to yield:where the value of g is the sigmoid function as follows:Therefore the function h transforms to — Now in case you are wondering why is this function taken for the value of g and why we could not have taken some other function, the answer is — the major advantage of using the logit function is that you get the simplicity of the methodology of linear regression without the disadvantages. Which means independent variables don’t have to be normally distributed or have equal variance in each group. This is because of interesting properties of the number e and I would highly recommend you to look at this video to gain a little bit of insight into the beauty of e. Hence, for classification where the data is binary, it is the ideal choice.Python ImplementationThe heart of logistic regression is the sigmoid function and that is what we will define at first.view rawsigmoid.ipynb hosted with ❤ by GitHubThe hypothesis(h) is the same as linear regression as in equation 2. The only difference is that instead of z we define the sigmoid of z as in equation 4.view rawlogistic_regression_hypothesis.ipynb hosted with ❤ by GitHubNow we need to define the error function. The error function needs to be such that the ‘punishment’ due to more errors should push the gradient of the function towards the minimum value. This is done using the Cost_function where we are basically finding the difference between actual and predicted value, and taking the logarithm of it. Here Y can take values of 0 or 1. Another fact to keep in mind is that logarithm of x with x=1 is 0 with the value approaching negative infinity as x approaches 0. Now, if Y is 1 then we just take the logarithm of the hypothesis and if Y = 0 then take the difference of the hypothesis from 1. Notice that if h is 1 or close to 1 when Y is 0 it will make the error big. Thus, a huge penalty will be introduced. Then we just take the sum of the errors and say this is our cumulative error.view rawcost_function.ipynb hosted with ❤ by GitHubNow we need to take the derivative of the cost function. This is the basis of the gradient descent algorithm. We will push the derivative towards 0 which means that the cost_function will be pushed towards the minimum value and the result will be that the hypothesis should approach the true value which is Y.view rawgradient_descent.ipynb hosted with ❤ by GitHubCompiling all the above ideas we write a high level logistic regression function that will take the X and Y as input and some other hyper parameters. The logistic regression function shown here is based on number of iterations for faster execution but you can easily change the logic where the cost is below some threshold.view rawhigh_level_function.ipynb hosted with ❤ by GitHub 

Learn More >

Master Data Visualization Tools and Techniques

We are a highly visual species. Visuals speak to us better than numbers and charts. Data visualisation is an easy and concise manner to convey ideas in a universal manner. Having the knowledge of a plethora of tools will enable us to present our ideas in the best way possible and help us be the most effective.In this post we will take a look at some of the common plots and make use of tools such as matplotlib, seaborn and some other visualization frameworks to view these plots.Line plotsThe most common plot is probably the line plot. In matplotlib the default plot is the line plot which can be shown using the command plt.plot(x_values, y_values). Below is an example of the same.view rawline_plots.ipynb hosted with ❤ by GitHubHistogramThe histogram is same as the line plots. They also show the distribution of data. Now the question is, when do we show line plots and when do we show histograms. According to me, histograms are a better choice when there is a lot of random swings in the data. And I think line plots shine when the changes in data across the spectrum is more smooth. Let’s take an example of a histogram.view rawhistogram.ipynb hosted with ❤ by GitHubBox plotA box plot can be used if the underlying grouping of the data need to be seen. You get an idea about how far spaced out are the quartiles in the overall distribution and where are the outliers if they are present. Also they are non parametric and do not make any assumptions of the underlying distribution of the data. Below is an example of the box plot.view rawbox_plot.ipynb hosted with ❤ by GitHubViolin plotsViolin plots are like the next level of histograms while they have the advantages of the box plots. So they show the probability density distribution of the data at different values. And like box plots, violin plots can be used to show the distribution of different categories of data. Thus, they easily show the differences between two similar groups of data. Take a look at the below example of a violin plot.view rawviolin_plots.ipynb hosted with ❤ by GitHub3D plotsSometimes some combinations of variables are what is important and hence, you will need to show them in a 3d space. As human beings, of course, we cannot scale this and take this to more than 3d space as we are not able to visualize multiple dimensions simultaneously. Below is the code to show components in 3d.view rawthreeD.ipynb hosted with ❤ by GitHubIn case you are interested in showing visualizations and the relationships between components where the vector space is more than 3d then take a look at this 3Blue1Brown video. The idea that is described in the video is known as parallel coordinates and is a widely used technique for showing multidimensional data. The code to implement your own parallel coordinates visualizations can be seen in this link.Showing relationships using graphsThe plots discussed above work best when you have structured data. But in some cases there are a lot of unstructured data with interconnected relationships between them (for example Twitter or Facebook user connections). Graphs are a good way to show such interconnected relationships between different types of data. Code to show the graphs and the relationships is shown here.GeographicalIf you are crunching data that is linked to spatial locations, you probably will need to show the distribution linked to some maps based on your target location. The preferred format for parsing location based information is the geojson file format. In python, geopandas is a popular library for plotting geographical information. In the below code the districts are color coded according to the states.view rawindia_geography.ipynb hosted with ❤ by GitHubReferences:A list of awesome visualisations in D3.List of visualisations.google sankey diagramsgeopandas visualisations.

Learn More >

Unlocking Reactive Programming: A Quick Overview

Reactive programming basically means relying on entities emitting elements, subscribing to its changes, and reacting to those changes, thereby saving you from having to re-check states and manually putting conditions in your code. It remarkably reduces the code lines and makes the code cleaner and easier to manage. Most importantly, it manages asynchronous code execution in a very elegant manner owing to its observable-subscriber method.This blog article will give you a quick skim of elements of Reactive programming in under 4 minutes. Here are some basic terminologies that are used while writing code using the Reactive Programming paradigm.Observable: It is an entity that can be of any data type and emits elements to its subscribers on the happening of every different type of event. Example: it emits error on an error event, emits the received value on an onNext event, etc.Observer: The Observer subscribes to observables and listens for changes in them and takes the programmed action on any change.Dispose Bag: A dispose bag holds disposables and is used for garbage management among the files. When subscriptions are created, they are added to a global dispose bag in a file. So whenever that particular files’ memory is deallocated, the subscriptions are properly disposed off from the Dispose bag thereby ensuring no memory leaks in the project.Subjects: Subjects have a dual nature i.e., as an observable and as an observer. There are four types of subjects :Publish Subject: Starts empty and only emits new elements to its subscribers. On completion, it emits completion to its new subscribers.Behavior Subject: Starts with an initial value and replays it or the latest element to new subscribers.Replay Subject: Initialized with buffer size and will maintain a buffer of elements up to that size and replay it to new subscribers.Async Subject: Emits only the last next event in the sequence, and only when the subject receives a completed event.Relays: Wraps their respective subjects, but only to accept and relay the next events. You cannot add a completed or error event onto relays at all, so they’re great for non-terminating sequences. There are two types of relays: PublishRelay and BehaviorRelay.Operators: there are different types of operators such as filtering operators, transforming operators, time-based operators which can be chained on observables using the dot operator so that the input from the observable can be modified as per requirement.Filtering operators: these operators filter out the results received from the data source. There are different types of filtering to do on data using different operators, for example, .elementAt(index): it gives the element at a particular index, .skip(n): it skips n elements emitted by data source, etc.Transforming operators: these operators transform data received from the data source into a different type. Some examples of this are .toArray(): this operator is used to transform the received elements into an array, .map(): allows you to enter a data type and change it to another data type.Combining operators: these operators combine results from observables. Example: .concat(): it is a static function that combines two observable sequences, .zip: it is used to couple two operations/requests and aggregate a result once both are done.Time-based operators: these operators are used to do operations like buffering and replaying data. Example: replay(): This operator creates a new sequence which records the last ’n’ emitted by the source observable, interval(schedulers: ): it produces an infinite observable sequence of Int values (effectively a counter) sent at the selected interval on the specified scheduler.Schedulers: A scheduler is a context on which a process takes place. So schedulers here are used to shift subscription contexts. There are different types of schedulers like MainScheduler, Concurrent Schedulers, etc. To Switch Schedulers, we use these two operators:subscribeOn(): we can switch threads by using this operator.observeOn(): this operator changes the scheduler where the observation happens.About the AuthorSmriti Arora is a young and passionate software development engineer and technical blogger with a knack for simplifying the technologies she works on for her readers. When she’s not writing or coding, she can found with loud music, setting the scene as she scours the internet in for tasty recipes and unique things to explore.

Learn More >

Mastering Statistical Data Science Fundamentals

Data Science, a discipline that has been gaining traction has now become one of the most sought-after career roles across the IT sector. As this particular career path faces a boom, so does the demand for data scientists across the world. With a steady incline on supply and demand on both sides, companies actively filter through scientists, checking for their mastery of the subject and also the quality of work that they can provide.While a career in data science by itself is not an easy feat to achieve (this being true even with the increasing demand), extra credits are given to those who have a sound knowledge of various statistical concepts.For those not familiar, Statistics can be easily defined as the study of the collection, analysis, and interpretation of data. Statistics by itself opens several doors when it comes to learning how to work with data. The study of data science by itself offers a good understanding of statistics to help aspirants to collect and analyze different kinds of data and perform operations on them to analyze and later report their findings.Data Scientists are taught to incorporate statistical formulas and derivations into certain machine learning algorithms to work upon and later generate certain patterns or trends from the data sets that they are using. The analysis that they provide generates immense value for the organizations that they work for/ clientele that they cater to. In a way, it is a given that statistics form the foundation upon which excellent data scientists are built upon.While most data analysts or scientists might be able to understand statistical concepts in terms of usage, knowing them in-depth and establishing well-rounded pillars in the basics will allow scientists to solve problems and provide deeper insights better than their regular counterparts.It is for this reason that we have made a checklist of sorts to help you understand and work out the various concepts that you might need to tackle!Statistical FeaturesData exploration stands to be one of the important duties of a data scientist. This exploration is usually achieved using tools created out of statistical features that help in the organization and searching of the data. Apart from these primary operations, statistical features play an important role in ancillary operations such as organizing the data and finding the minimum and maximum values, finding the median value, and identifying the quartiles.Descriptive StatisticsDescriptive Statistics is often used to showcase the data that you are working on in a concise and meaningful way. It provides you with descriptive summaries of the data and allows data visualizations easily. They are mostly employed when dealing with untamed and raw data that presents a challenge when it comes to reviewing or communicating. Descriptive statistics are used to show data as it is rather than show information that can be obtained from it.Probability TheoryProbability refers to chance or the likelihood of an event (any event) occurring. It mathematically measures the chances that something might occur and gives a quantifiable number between 0 and 1. Data Scientists need to understand probability theory and how it works to work out algorithms that might give them a chance of looking at what might be the probable things that might happen when working with a large amount of data. Probability theory by itself boasts a number of formulas that can help derive mathematical figures of outcomes which might help data scientists dig their way out of tough computations on data.Bayesian StatisticsBayes Theorem is defined as: “In the Bayesian paradigm, current knowledge about the model parameters is expressed by placing a probability distribution on the parameters, called the prior distribution.”Prior distribution here indicates a scientist’s current knowledge on a subject. Bayesian statistics or thinking deals with the occurrence or arrival of new data and later updating beliefs on them.This new information is looked upon as a likelihood and is combined with the previous one to produce an updated probability distribution called the posterior distribution.Probability DistributionsProbability Distributions are mainly used by data scientists to measure and calculate the likelihood of certain values or events occurring. By definition, it is all the various possible distributions or outcomes of a random variable and their corresponding probability values between zero and one.Dimensionality ReductionData Scientists often find it cumbersome to deal with extremely heavy or factor-rich data sets. Such data sets limit scientists from obtaining precise and actionable insights and are often a hindrance than an aid. Dimensionality reduction serves as a beacon of hope and helps scientists reduce the dimensions of these data sets and the overall complexity of the analysis. It also aids in faster computation and helps scientists develop more precise and accurate models.ResamplingAt times, a data scientist or other statistician might encounter a data set that happens to be irregular or not inherently balanced. To counteract this, the handlers of data use a simple technique to alter and balance the unequal data sets. This technique is also referred to as resampling. Resampling constitutes two forms, over and undersampling. Oversampling is used when there is a lack of data in the data sets and Undersampling is used to remove redundant data and change the focus onto the more prominent ones.

Learn More >

Retail Chatbots: Boosting Sales and Customer Satisfaction

The retail industry has long been at the forefront of adopting new technologies to improve the customer experience. One such technology that has gained popularity in recent years is the use of retail chatbots.Chatbots are computer programs that use artificial intelligence to carry out conversations with human users. They are designed to be able to understand natural language and respond to queries and requests in a way that mimics human conversation.In the retail industry, chatbots are being used to assist customers in a variety of ways. For example, they can be used to provide product information and recommendations, help customers place orders, and assist with returns and exchanges.One of the key benefits of chatbots in the retail industry is that they are available 24/7. This means that customers can get assistance at any time of day, regardless of whether or not there are human customer service representatives available. This can help to improve the overall customer experience, as customers are able to get the help they need quickly and efficiently.They are also able to handle multiple customer inquiries at once, providing immediate responses and eliminating the need for customers to wait on hold or be put on hold. This ensures that customers receive timely and accurate information, which can help to improve customer satisfaction and reduce customer churn.Additionally, chatbots are able to provide personalized recommendations and suggestions based on a customer’s previous interactions and browsing history. This can help retailers to build stronger relationships with their customers and increase the likelihood of repeat purchases.Another key benefit of chatbots in retail is their ability to handle simple, repetitive tasks. For example, a chatbot can be used to handle common customer inquiries such as order tracking, product availability, and shipping information. This frees up customer service teams to focus on more complex and high-value tasks, such as resolving customer complaints and addressing technical issues.Furthermore, chatbots can be integrated with various platforms and systems, such as e-commerce websites, social media channels, and messaging apps. This allows retailers to provide a seamless customer experience across multiple touchpoints, which can improve customer engagement and loyalty.One of the most compelling benefits of chatbots in retail is their ability to drive sales. By providing personalized recommendations and suggestions, chatbots can help retailers to upsell and cross-sell products, which can increase average order value and revenue. In addition, chatbots can be used to offer personalized promotions and discounts, which can encourage customers to make a purchase.Overall, the benefits of chatbots in retail are clear. They are able to improve customer engagement, reduce the workload for customer service teams, and drive sales. As AI technology continues to evolve, we can expect to see more and more retailers adopting chatbots as a key part of their customer experience strategy.Explore How You Can Grow Your Retail Business Through Chat Bots With NineleapsWe at Nineleaps have extensive expertise in helping organizations like yours achieve their digital transformation journey. Our services are streamlined to help you anywhere in your digital journey while identifying and enhancing your digital capabilities.This article first appeared on LinkedIn.

Learn More >