Breaking into Product for Interns or Early Career PMs

moderate read, takes 4 mins

I woke up this fine spring morning. With a jet lag and quarantine mandate, I decided to look up LinkedIn at 4.30 am. Only to realize that I have over 100 unread messages. At least 50 of those asked for advice on breaking into a PM internship.

So, with newfound motivation to restart my blog, I started typing this out. Truly candid, honest opinions, and personal experience all rolled into one post.

Internship/ Early career PM search advice from someone who was looking for internships not so long ago. (hint: c’est moi!)

Here’s what helped me. It’s a very personal take and this may differ for various people in different stages of their search. I hope that helps you all too!

Me, in 2017, trying to figure out what it takes to be a good PM

Things to do:

– Reflect
Start by thinking about what you want out of the role. This could be early experience in product, getting a good brand on your resume, learning to be a generalist PM, or trying out various specializations. Honestly, you can’t go wrong with this one It’s about you and nobody can question that.

– Figure out what interests you
Think through various products that you like and narrow down a list of products that you would LOVE to build. Ex: B2B SaaS, cPaaS, consumer products, educational products, streaming services, AI products, social networks, etc.

– Question yourself
Once you narrow down the types of products that you’d LOVE to work on, you should craft stories. Essentially, question yourself on WHY you’d like to work on this, why does this stand out, are there any personal experiences that would make this product more desirable? Nothing matters more than when a candidate demonstrates their passion for products! (Of course, we can understand if it is genuine.)
Here’s a page I put together to craft a story for a job at Duolingo: https://kaivalyapowale.com/working-at-duolingo/

– Make a list
Now, get down to a list of companies OR just browse through LinkedIn to find companies. Even look for companies not hiring for interns right now. Once you have a target list, reach out to PMs and product leaders within that company (better if you can narrow it down to teams but I understand that can be difficult)

– Reach out
Start building relationships before you need help. This is great advice but not so helpful when you have 2 months to look for an internship. So, I’d suggest being direct: Intro, saw a nice role, about me/portfolio, why I wanna work, can you put me in touch/refer?, thank you for your time.
Here’s what I shared when I was looking to break in: https://kaivalyapowale.com/what-i-built/

– Be patient.
PMs are crazy loaded with work, the # of DMs they get every day, and trying to maintain a healthy balance in life. Message, follow-up, don’t spam or be rude (surprising how many people need to be told this). If you don’t hear back. That’s okay. Everyone needs to prioritize according to things that are important to them. And like any good PM, they should.

– Prepare
Once you land the initial interview, the stories you crafted and your passion through the process will get you through. (I am considering the fact that you’d do your own case preps and brush up on PM core skills)

Things to remember:

– Keep your head up
Being able to land an interview does not define your capabilities. It’s a matter of time before you land one. Not an “if” question but a “when” question.

– Do your own research
There’s a ton of wisdom out there about product management. If you seek, you shall Google and read yourself! Seriously. Setting up calls with PMs and asking them about product design case approach is not the best use of anyone’s time. (A better way would be to ask them for a mock case they loved or ask them where they fumbled and how they fixed it)

– A few PMs to follow
Lewis C Lin (no PM career prep can begin without his content), Clement Kao‘s blog, recently found a goldmine from Rohan Rajiv. Note that there are many many more folks out there and you’ll find many lists if you Google.

– Learning matters the most
Focus on core PM skills and find learning opportunities instead of growth opportunities. As an early PM (or an enthusiast), see how you can learn new things, experiment around with product areas, and question every task that comes your way. Don’t seek titles that sound fancy. Although it’s cool to feel that way. We have all been there 🙂

– A small step for you is a giant leap for your career
It may be easier and better to go for smaller companies or not-so-fancy brands. Not just the fact that it is easier to access people who work there but the reality that you’ll get to learn more in smaller teams.

– A good mix of work, passion, and planning will get you through the door.

Looking forward to hearing from you if this (and I) was of any help at all 🙂

Artificial Intelligence in Healthcare

a bit heavy, takes 6 mins

Having worked on a project aimed towards improving health and wellness across hospitals in the US, I grew closer to understanding the issues faced by healthcare systems. Automation of operational tasks and proper technical infrastructure can eliminate the vast majority of these issues. With this in mind, I explored the advancement of Artificial Intelligence in Healthcare, the problems it can solve, and a few roadblocks along the way.

Machine Learning is transforming the healthcare industry by changing the outlook on care delivery, operational optimization, and disease detection.

The Problems

Enterprises in healthcare have long been troubled by problems like maintenance of health records, identification of care programs, early disease diagnostic, insurance fraud, waste, and abuse (FWA), time spent on medical imaging, and outbreak prediction for diseases. They are now looking towards machine learning (ML) techniques and artificial intelligence (AI) systems for the solution.

The US Healthcare system alone generates approximately 1 trillion gigabytes of data annually. ^[1]

The ecosystem map for healthcare data by Datavant

This data is both structured and unstructured, creating the need for a complex set of algorithms to make sense from it. Typical statistical methods work on confined problems making use of structured data to drive insights. ML is required to learn from highly complex datasets and derive a relation between multiple parameters informing experts of the multimodal facets of the issue.

How to make sense of this data?

Various ML techniques apply to the healthcare industry based on the specific problem and the intended outcome.

1. Disease detection and prediction uses supervised learning techniques where the model is trained on a set of target outcomes. This technique is implemented with expert opinions and structured datasets to predict the occurrence of diseases or ailments.

2. Medical image analysis is implemented using Computer Vision and deep neural networks (like CNN) as they include highly complex and undescribed variables in the dataset.

3. Unsupervised learning algorithms find their application in outbreak prediction as they can be used for clustering and anomaly detection.

4. Insurance fraud and patient record maintenance use techniques like Natural Language Processing (NLP) and deep learning to make sense from unstructured patient records and missing data.

Where is the value for AI?

AI in Healthcare is expected to grow at a CAGR of 50.2% from 2018 to 2026 and reach a market size of $150 billion. ^[2]

ML is projected to deliver 63% of that value. ^[3]

The specific use cases of ML that can be used to solve the problems in healthcare include Automation of Patient Health Records, Disease Outbreak Prevention, Early Detection of Chronic Diseases, Patient Journey Monitoring, Equipment Maintenance, Operational Scheduling, Fraud Detection, and Medical Imaging Diagnosis.

Top 2 Use Cases

1. Early Detection of Chronic Diseases

The total annual cost of chronic diseases in the US is $3.7 trillion which is close to one-fifth of the entire US economy.

This cost is expected to increase as the US population is aging. ^[4] Using ML and predictive analytics can help identify high-risk patients and develop tailored monitoring or care programs that can prevent a total cost of $30.8 billion related to chronic diseases. ^[5]

2. Operations Scheduling

Deep neural networks can be used for the optimization of operational scheduling by leveraging electronic health record (EHR) data and resource utilization patterns.

This alone can save USD 500k per Operation Room per year. ^[6]

Operational Scheduling will allow for better patient care and will ensure the availability of resources including nurses and staff.

Necessary Infrastructure

ML and Deep Neural Networks are expected to reach the plateau of productivity in the next 2-5 years. To maximize the value we can create through them, we need to have the infrastructure and human capabilities in place. The technological infrastructure as it pertains to these two use cases includes proper data collection and storage techniques, hardware capacity to train and deploy ML models, and an interconnected system of equipment (IoT).
An ML model is only as good as the data provided to it. The sophistication of data collection techniques is important to realize a sound and scalable AI system. The lack of high-quality data and ineffective privacy protection are hindering the growth of AI in healthcare. ^[7]

Readying the Human Capital

Not a black box. Interpretabe and usable by doctors.

Healthcare is different from other industries in the sense that there is an intricate and intimate relationship between healthcare providers and patients. The ML systems and algorithms in healthcare need to be transparent where the various metrics and decision criteria are clearly understood by the doctors or providers. This means the human capital must be data literate and trained about the impact of ML models on their decision chain. This will be a key factor in the acceptance of ML in healthcare. ^[1]

Photo by Stephen Dawson on Unsplash

Delivering value through AI

Current players in the market offer a comprehensive set of products that can be implemented in the healthcare industry to deliver value for these use cases. They should look to serve the industry using the Three Horizon Model i.e. through their current capabilities, emerging technologies, and future ventures.

1. Growing the current business
ML techniques that have applications like Predictive Maintenance of equipment, Sensor Health monitoring, and Continuous Analytics can deliver value for Operations Scheduling. These applications help monitor the systems and predict asset failure that results in lower downtimes. Machine Learning services and AI modeling platforms will empower organizations to deploy models that help optimal predictions for chronic diseases.

2. Emerging Opportunities
The second horizon for healthcare AI business is its emerging opportunities that will create value shortly and will require considerable investment. This includes investing in Natural Language Processing, Computer Vision, and Speech Recognition to deliver use cases like Medical Imaging Diagnostics, Patient Journey Monitoring, and Improved Patient Record Maintenance.

3. Imagining the Future
The third horizon contains planning for profitable growth down the line including partnerships with upcoming tech labs, diversification of resources and focus areas. The future techniques will go beyond the focus on resolving operational issues to delivering value to the healthcare industry through Personalized Care.

Anticipating the future needs of the industry and preparing for acceptance of ML will empower us to benefit from the advent of AI in Healthcare.

References:
[1] https://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/machine-learning-and-therapeutics-2-0-avoiding-hype-realizing-potential
[2] https://www.accenture.com/fi-en/insight-artificial-intelligence-healthcare
[3] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6616181/
[4] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6429690/
[5] https://hbr.org/2017/05/how-machine-learning-is-helping-us-predict-heart-disease-and-diabetes
[6] https://leantaas.com/wp-content/uploads/2019/01/OR-case-study-booklet-27-01_online.pdf
[7] https://www.cbinsights.com/research/report/ai-trends-healthcare/

Eminem’s album trends and Music To Be Murdered By (2020)

From asking for a change in gun laws to calling out shootings at an Ariana Grande concert, Eminem’s lyrics in Music To Be Murdered By have been surrounded by controversies since the day it launched last week. Yet, this is not unusual for anyone familiar with Eminem’s work. He is popular for lyrics that do not conform to political correctness and swears that will make your mom upset.

Just a week later and Music To Be Murdered By has topped the Billboard 200 making Eminem the only artist having 10 studio albums that topped the chart. The times have changed since Em first launched his studio album. Let’s see if his music has changed too!

In this one, I have covered the trends for Eminem’s 10 studio albums. This includes audio analysis, changing feature trends with albums, and a look at the positivity of Em’s albums.

I have primarily used data from Spotify’s API using multiple endpoints for albums and tracks. I supplemented the data with stats from Billboard and calculations from this post.

Trends in Positivity for all albums

I created a dashboard that shows the average Positivity of Eminem’s albums over the years. This feature was obtained from Spotify’s Web API endpoint. Spotify calls this metric Valence which reflects on positivity (hence I have called it that to simplify things). Here’s how Spotify has defined this metric
“A measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).”

Every point on the plot represents one song in the album.

It is clear that over the years Em’s songs are perceived to be more and more negative. Music To Be Murdered By is by far the most negatively perceived album according to the audio analysis. And it seems so too in reality. With songs like ‘Darkness’ and ‘In Too Deep’, Eminem has addressed a lot of sadness around in this one.

The chart above tells us that, in general, the Positivity increases with an increase in Acousticness, Loudness, Energy, and Danceability of the songs. That makes sense as more acoustic and danceable songs are the effect of pop collaborations whereas louder and more energetic songs tend to sound happier.

Here’s how the Positivity trends with respect to Speechiness of the songs i.e. with an increase in words, the songs are more positive. That sounds unusual and pretty much unlike Em!

This is not true for all albums though. Whereas Positivity increases with Marshall’s words in his latest, for albums like Relapse the positivity actually decreases with more words.

It still seems highly unusual for Eminem’s words to be perceived more positively. Let’s see how have those words changed over time!

Words and Song Durations

Over time, the album duration is going down which shows that Eminem is making shorter albums. That’s for sure with just 13 songs in Kamikaze as compared to an average of 19 songs per album for others.

When we look at song duration, it tells the same story. The song duration has gone down over the years. This is a phenomenon affecting the industry as an influence of pop culture.
It is worth noting that the song duration bumped around Marshall Mathers LP 2. This album was Em’s comeback album and featured longer songs like ‘Rap God’ and included a couple of skits pulling up the average duration.

Another possible explanation is that the song duration is decreasing as Eminem is spittin’ out words faster and faster with every release. This is shown the likes of ‘Rap God’ in MMLP2 and now in ‘Godzilla’ with Em making history by packing a whopping 7.23 words per second!

Eminem swears like he wants to upset your mom. His work is widely popular for being laden with swear words, controversial, offensive, and misogynistic lyrics. Let’s see how the swear words in his songs have changed over time!

Eminem’s most controversial work has been during the early 2000s. This chart is a proof of that. The number of swear words used has gone down over time except for the uptick in Kamikaze. Over the years, he has calmed down and made a few amends; like forgiving his mom.

Speechiness shows that the ratio of words to music has also gown down over the years. As both the trend lines suggest, Eminem is using lesser words, more music, and definitely lesser swear words. Would that make your mom happy?

Features over the years

A major part of Spotify’s API is the audio feature analysis by its music intelligence company The Echo Nest. This data is calculated through its proprietary algorithm and not much is shared about how the features are calculated.

These features give a great overview of a song, album, or playlist though. It is worthwhile to take a look at them. So, I went ahead and created a simple dashboard for you to explore all the features firsthand! Take a look.

That’s it for this one! I will be sharing the code to acquire this data soon. Until then, learn how to access Spotify’s API or check out my GitHub to see if I have uploaded the code to my repo. Thanks for reading!

Update 05/11/2020: You can download the Python notebook and data directly from here. The notebook has all the guidelines for the project.
You can also go to my GitHub for more instructions. You can return here to look at visualizations!

Download Jupyter notebook

How to access Spotify API

Spotify’s Web API is a RESTful API and gives you access to music artists, tracks, albums, public playlists, and user-specific data. The data access depends on the authentication you acquire.

So, I am going to write about acquiring the Authentication and Validation before I get to my project or analysis.

This process is quite similar to Accessing Twitch API or any other RESTful API for that matter.

1. Create an account with Soptify for Developers at https://developer.spotify.com/. Go to your dashboard and register a new app.

2. Enter basic information and the purpose of your app. Once you do this you will get your Client ID and Client Secret.

3. Check out Spotify’s Web API documentation to get a complete understanding of all the authentication flows and endpoints. Or stay tuned as I am going to cover it in my upcoming pieces.

I personally refer to the API Reference. It’s quite simple to understand and gives a comprehensive view of everything you need to get started.

4. Authenticating ourself to get the Access Token

I am using the Client Credentials flow because we do not need any user’s data as of now. Spotify’s Access Token’s last for 3600 seconds i.e. 1 hour.

	import pandas as pd
	import requests
	import json
	from pandas.io.json import json_normalize
	import time
	import base64

view raw import_libraries_spotify_auth.py hosted with ❤ by GitHub

To authenticate, you need to encode your Client ID and Client Secret in your endpoint request. I am going to write a function that takes care of encoding and the request.

	# Defining base64 encoding of the IDs
	def base64_encode(client_id,client_secret):
	encodedData = base64.b64encode(bytes(f"{client_id}:{client_secret}", "ISO-8859-1")).decode("ascii")
	authorization_header_string = f"{encodedData}"
	return(authorization_header_string)

view raw base64_encode_spotify.py hosted with ❤ by GitHub

You can call the function and pass the IDs as arguments but hold on. We can write another function directly for the request and call the base64-encode within that.

	def accesstoken(client_id, client_secret):
	header_string= base64_encode(client_id,client_secret)
	headers = {
	'Authorization': 'Basic '+header_string,
	}

	data = {
	'grant_type': 'client_credentials'
	}

	response = requests.post('https://accounts.spotify.com/api/token', headers=headers, data=data)
	access_token = json.loads(response.text)
	access_token = access_token['access_token']
	return(access_token)

	access_token = accesstoken('your Client ID','your Client Secret')
	access_token

	"""

	Sample response is

	{'access_token': 'BQA5Zu1uNhJbKpr3tBcWRseAy-qfwwPXMjJQEvXyqKdy0Y1XaQvsC8HTE7qYuI1e_fMUVuwLltADeA-QuNc', 'token_type': 'Bearer', 'expires_in': 3600, 'scope': ''}

	And using this function you don't need to worry about extracting the access_token from the JSON response.
	The accesstoken function will return a string like this

	'BQA5Zu1uNhJbKpr3tBcWRseAy-qfwwPXMjJQEvXyqKdy0Y1XaQvsC8HTE7qYuI1e_fMUVuwLltADeA-QuNc'
	"""

view raw access_token_spotify.py hosted with ❤ by GitHub

Now you have your access token. You can directly use the variable access_token in all other endpoint requests.
A sample request looks like this

That’s all for this one. I am going to be writing more and will present some examples of the API. Stay tuned.

Twitch Live Dashboard – Accessing Twitch API

What is Twitch?

Twitch is the world’s largest live streaming platform focused primarily towards gamers and e-sports. With an average of 15 million unique daily viewers, the world spent a whopping 560 billion minutes watching content on Twitch in 2018. [1]
We decided to see take a look at the Top Games and Top Streamers being watched on Twitch. This was made easy by Twitch as they provide a ton of developers tools including relevant data accessible through Twitch API.

Here’s a pretty simple dashboard we created on Tableau

You can take a look at an interactive version published on my Tableau Public profile. This dashboard shows Top Games streamed and Top Streamers based on the number of Viewers. If you select a game then you can see the Top Streamers for a particular game. Tableau Public only takes data extracts so this public dashboard is not pulling live data from Twitch.

Link to the GitHub repo: https://github.com/kaivalyapowale/Twitch-Dashboard/

Team project by Vardayini Sharma, Maddhujeet Chandra, and Kaivalya Powale.

Tutorial

I have outlined a complete tutorial from accessing Twitch API using a python script to the final Tableau Dashboard design.
(For our real-time dashboard we used an Amazon Web Services EC2 instance to run our python script and AWS RDS to store the live data. I am not going over those in this tutorial. Instead, I am using a locally stored csv file which my python script updates automatically every 60 seconds.)

1. Create a Twitch Developer account at https://dev.twitch.tv/. Go to your Dashboard and ‘Register a New Application’. You will need this to get your Client ID and Client Secret.

Use localhost as redirect link. You will get your Client ID and Client Secret on this page.

2. Check out the Twitch API documentation to fully understand the API calls, what data it offers, and what authentication it requires for the data you need.

Twitch API has a lot of available requests. Use: https://dev.twitch.tv/docs/api/reference

3. Authenticating ourselves to getting the Access Token.

I used a Jupyter Notebook to write my python script. You can use whatever you are comfortable with. For beginners, I’d recommend Jupyter Notebooks. (do some research to see what best fits your skill level)

First begin by importing the necessary libraries

	import json
	import requests
	import pandas as pd
	from pandas.io.json import json_normalize
	import time
	import threading

view raw import_libraries_twitch.py hosted with ❤ by GitHub

Now move onto authentication to get the access token

	#Client ID: –***—
	#Client Secret: —***—
	#Client ID and Client Secret are sensititve and you should not share them

	client_id= <yourclientid>
	client_secret= <yourclientsecret>

	#Request for the access code using requests library
	#I have chosen this method of authentication with my goal in mind

	access_code = requests.post('https://id.twitch.tv/oauth2/token?client_id='+str(client_id)+'&client_secret='+str(client_secret)+'&grant_type=client_credentials')

	#access token response is a JSON-encoded app access token
	access_token = json.loads(access_code.text)
	access_token = access_token['access_token']

	#Sample response is
	"""
	{
	"access_token": "prau3ol6mg5glgek8m89ec2s9q5i3i",
	"refresh_token": "",
	"expires_in": 3600,
	"scope": [],
	"token_type": "bearer"
	}
	"""

view raw authentication.py hosted with ❤ by GitHub

4. We need two types of API calls for this dashboard Get Top Games and Get Streams.

We will first access the API for Top 100 Games by number of viewers. Using the Game IDs for these games, we will get Stream data for them.

	# Getting data for Top 100 Games by number of viewers
	# Default response is for 20 games so you will have to set the parameter 'first to 100'

	headers = {
	'Authorization' : 'Bearer '+str(access_token),
	}
	games_response = requests.get('https://api.twitch.tv/helix/games/top?first=100', headers=headers)

	# The response will be a JSON which will include the response data and the pagination cursor
	# We need to extract the data from the JSON and convert it into a pandas dataframe

	games_response_json = json.loads(games_response.text)
	topgames_data = games_response_json['data']

	# Converting to a pandas dataframe
	topgames_df = pd.DataFrame.from_dict(json_normalize(topgames_data), orient='columns')

	# See the first few lines. The response includes id, name, and box art url for the game
	topgames_df.head()

view raw get_top_games_twitch.py hosted with ❤ by GitHub

To get the Top Streams for these games we will have to pass the game IDs as strings in the API call one at a time. For this, we need to create a FOR loop to get data for all the Games.

	# I am getting only the top 25 streamers for the first game
	headers = {
	'Authorization' : 'Bearer '+str(access_token),
	}
	topstreamsforgame_response = requests.get('https://api.twitch.tv/helix/streams?game_id='+str(topgames_df%5B'id'%5D%5B0%5D)+'&first=25', headers=headers)

	# Load the JSON
	topstreamsforgame_response_json = json.loads(topstreamsforgame_response.text)

	# Extracting data from the JSON
	topstreamsforgame_data = topstreamsforgame_response_json['data']

	# Converting into a DataFrame
	topstreamsforgame_df = pd.DataFrame.from_dict(json_normalize(topstreamsforgame_data), orient='columns')

	# FOR loop to get top 25 streamers for rest of the games in our list
	# To keep the dashboard lightweight and relevant, I am using only the Top 20 Games and Top 25 Streamers per game

	for i in range(1,19) :

	headers = {
	'Authorization' : 'Bearer '+str(access_token),
	}
	topstreamsforgame_response = requests.get('https://api.twitch.tv/helix/streams?game_id='+str(topgames_df%5B'id'%5D%5Bi%5D)+'&first=25', headers=headers)

	topstreamsforgame_response_json = json.loads(topstreamsforgame_response.text)
	topstreamsforgame_data = topstreamsforgame_response_json['data']
	topstreamsforgame_df_temp = pd.DataFrame.from_dict(json_normalize(topstreamsforgame_data), orient='columns')

	frames = [topstreamsforgame_df, topstreamsforgame_df_temp]
	topstreamsforgame_df = pd.concat(frames, ignore_index=True)

	# Look at the data we retrieved
	topstreamsforgame_df.info()

view raw top_streams_twitch.py hosted with ❤ by GitHub

Now, for the final trick, we will define a function which will enclose all our code and put a Timer so that it pulls the data every 60 seconds.

	def twitch():
	threading.Timer(60.0, twitch).start()

	# Top Games
	headers = {
	'Authorization' : 'Bearer '+str(access_token),
	}
	games_response = requests.get('https://api.twitch.tv/helix/games/top?first=100', headers=headers)

	games_response_json = json.loads(games_response.text)
	topgames_data = games_response_json['data']
	topgames_df = pd.DataFrame.from_dict(json_normalize(topgames_data), orient='columns')

	# Top Streamers
	headers = {
	'Authorization' : 'Bearer '+str(access_token),
	}
	topstreamsforgame_response = requests.get('https://api.twitch.tv/helix/streams?game_id='+str(topgames_df%5B'id'%5D%5B0%5D)+'&first=25', headers=headers)

	topstreamsforgame_response_json = json.loads(topstreamsforgame_response.text)
	topstreamsforgame_data = topstreamsforgame_response_json['data']
	topstreamsforgame_df = pd.DataFrame.from_dict(json_normalize(topstreamsforgame_data), orient='columns')

	for i in range(1,19) :

	headers = {
	'Authorization' : 'Bearer '+str(access_token),
	}
	topstreamsforgame_response = requests.get('https://api.twitch.tv/helix/streams?game_id='+str(topgames_df%5B'id'%5D%5Bi%5D)+'&first=25', headers=headers)

	topstreamsforgame_response_json = json.loads(topstreamsforgame_response.text)
	topstreamsforgame_data = topstreamsforgame_response_json['data']
	topstreamsforgame_df_temp = pd.DataFrame.from_dict(json_normalize(topstreamsforgame_data), orient='columns')

	frames = [topstreamsforgame_df, topstreamsforgame_df_temp]
	topstreamsforgame_df = pd.concat(frames, ignore_index=True)

	# Now that the FOR loop is exited and we have all our data, we export it into a csv

	export_topgames_csv = topgames_df.to_csv (r'<filepath>.csv', index = None, header=True) #Don't forget to add '.csv' at the end of the path

	export_topstreamsforgame_csv = topstreamsforgame_df.to_csv (r'<filepath>.csv', index = None, header=True)

	# Our function is defined and it overwrites the CSV every 60 seconds. Now, we call it.
	twitch()

view raw final_dashboard_twitch.py hosted with ❤ by GitHub

As you can see, I have exported the csv within the function. So, it updates automatically every 60 seconds. Now, we have to connect it to Tableau and make our Dashboard.

5. Creating the Dashboard on Tableau

Open Tableau and connect the csv file for Top Games. Pull the Top Streams csv and create an inner join on ‘Id = Game Id”. Ensure that you have a live connection with the data source.

Now that the connection is made, go over to sheet 1. Pull the Viewer Count from the Measures into the Sheet and pull the Name into the Rows section. Make it a bar chart and order it.

Make another Sheet for top Streamers.

For the Dashboard, pull in both the sheets. And set the Games Sheet as a filter for the Streamers sheet. This will show you overall Top Streamers and game-wise Top Streamers.

This is our final dashboard. After this, I formatted it to make it look prettier and match Twitch’s design guide.
Set background to black i.e. HEX #000000 and the bar colors to Twitch purple i.e. HEX #6441a5

Remember, our data is updating live so we need to set the dashboard to auto-refresh every 60 seconds or refresh it manually.

6. Setting auto-refresh

I set the auto-refresh to update the dashboard every 60 seconds. This can be done using an auto-refresh algorithm or something like an AutoHotkey. Use Cmd+R to refresh the data source on mac and F5 on Windows. I have attached a file for auto refresh on Windows.

Download Jupyter Notebook + Autohotkey

That’s it. Hope this was helpful. Please get in touch if you have any recommendations or doubts. Thanks for reading through!

References:
[1] https://www.businessofapps.com/data/twitch-statistics/

Starting out with Agile.

light-read, takes 4 mins

Why should you read this?

When I started learning about Agile, I was confused on so many levels. There’s a lot of content out there and I had a tough time tracking through the fundamentals without getting too deep into it.
After knowing I was not the only one, I thought of introducing the basics as a stepping stone for people to go out and explore!

These are the very fundamentals in an easy-to-consume form, taken out directly from my first notes on Agile!

Doesn’t matter if you are into design, engineering, or business, you will cross paths with Agile sooner or later. So, it is definitely worth knowing the basic terms. (trust me I won’t get into the details in this one!)

What is Agile?

Agile is a development approach that focuses on delivery.

Traditionally, Agile was used for projects relevant to software development. With its widespread acceptance, it is used across industries to get work done.
Agile ensures that teams adapt to dynamic environments and do what’s required, when it is required.

In Waterfall approach (all oldschool projects), you first plan on all that you want to do and you precisely decide what you are going to make before you even start working!
It is great but what happens if the thing you planned for is no longer required? That’s where Agile shines.

This image depicts the waterfall approach to projects. It is a linear approach where you need to finish all planning before you get working. — A lot of time is spent before working and the planning makes this approach rigid

In Agile projects, you never sit down to decide what your final output should look like. You focus on what is the highest priority right now; and you do it! With time the goal may be constant or maybe not. One thing will be constant for sure, faster delivery.

In Agile, too, you plan work but you plan for shorter goals so that you deliver small things faster

The most important thing to remember is that your final scope is not defined. Scoping is done on a regular basis to re-prioritize our current work.

Agile is not a method, it is a way of thinking. Thinking what needs to be done and figuring out how we do it fast. The value is in learning quickly.

Learn fast, deliver faster.

This way of thinking/mindset has been adapted and used in many Agile frameworks, most popular being Scrum and Kanban. These frameworks are primarily based on constraints (like a limit on something). In Scrum, the constraint is time you take to work and in Kanban it is how much work you do at any given time.

You know the very basic concepts now. Here’s 5 important terms you would need to know.

1. The Agile Manifesto

17 people, 68 words

In 2001, 17 people met (in Snowbird, Utah) to discuss and improve the software development process. The goal was to find a common ground and understand what enables a software development project to succeed.
They came up with 4 values documented in the Manifesto.

*The Snowbird 17’s Four values (in bold) that outline the Agile mindset.*

12 Principles of Agile Software were later built to expand on these values.

2. Scrum

An Agile framework with time as constraint

Scrum is the most popular Agile framework that focuses on planning and implementing work in iterations. These iterations are called sprints.

The whole project is broken down into definite timeframes (usually 2 weeks) in which you plan and complete the work. One timeframe corresponds to one sprint. After a sprint ends, you plan and start the next one.

3. Kanban

An Agile framework with Work-In-Progress as constraint

Kanban is the Agile framework that focuses on completing highest priority tasks as soon as possible. Hence, in Kanban, there is a restriction on how much work can be undertaken.

Tasks are put up on Kanban Boards and belong to either of the 3 columns: TO-DO, IN PROGRESS, DONE.
There’s a limit on how many tasks can be put in the IN PROGRESS column at a time.

*I put a limit of 3 tasks for WorkInProgress. Task 11 might be more important than 4, 5, and 6 but cannot be done until we finish either of those* three.

TO DO column is re-organized every day to have the highest priority tasks on top. These are supposed to be undertaken first.

4. Self-Organizing

In Agile, the developers self-assign tasks

An Agile team is always self-organized. That is the project manager (or product manager) do not assign responsibilities and load on the team. The PM only prioritizes what needs to get done first. The team pulls the tasks based on the list and their capabilities.

5. Standups

Daily update meetings

Almost all Agile teams have daily standups. As the name suggests, everyone stands (to keep meetings short and the team focused) to discuss about their work and if there are any issues that need to get resolved.

This is all I am going to cover in this one. We haven’t even scratched the surface yet. But don’t worry, I am soon coming out with a series that will tell you everything you need to know about Agile!
Let me know if you liked this one.

How Airbnb manages to stay innovative

Scenario (excerpt from audio above)

You are going to San Francisco for a conference and you need a place to stay
All the hotels on your radar are booked out and the ones that are available are either very high-end or a bit too shady!
In this situation, you’d either have to shell out extra cash or compromise on your safety.
Well, thankfully, you don’t have to do that. You could just get an Airbnb, an airbed and breakfast.
Airbnb lets you choose from hundreds of local houses for your stay.
This gives you a local experience of the region at cheap prices. So you live like a local; where you feel comfortable!

How does Airbnb do this? In an industry with cut-throat margins where competitors are striving to get profits, how does Airbnb continue to grow? How are they so innovative?

Our team analyzed Airbnb’s innovation strategy to understand how the company sustains its ability to capture the market and create a strong pull from customers.

I have tried to keep the content lightweight and easy-to-read. Let me know if you’d like to read the detailed version.

Let’s take a look at Airbnb’s foundations and current strategies.

The Past

Airbnb was conceived in 2007 when two roommates started renting out their living room to conference visitors in San Francisco. They provided guests with an airbed and breakfast (roll credits) for $80 per night.

At that time, the concept of renting out someone’s house was a thing that the industry had not seen before. It created a new section within the accommodation industry. It was something refreshing

… something like …

The Blue Ocean

According to the blue ocean strategy, a company needs to Eliminate, Reduce, Raise, and Create factors compared to the industry standards. And that’s what Airbnb did!

While reducing factors like prices & safety, Airbnb created a ubiquitous availability of home-like experiences.

Airbnb’s business model was developed on being a low-cost solution for a low-value customer. And using the concept of ‘Sharing Economy’, they created a newfound supply of income for the hosts. This led to a simultaneous rise in demand and supply of accommodation.

Disrupting the industry

They provided a service that was lower than the market standards and over time, they improved the quality-of-service which the mature market could not cope up with.

A typical disruptive innovation starts with a low-par service with respect to the industry

In this asset heavy industry where competitors were bogged down with owning and maintaining real estate, Airbnb owned just the intellectual property and leveraged the assets of their hosts to gain market share.

Barriers to Innovation

Barrier: Safety – how could guests be safe in strangers’ homes?
Solution: Risk scoring of properties, watchlist and background checks, safety workshops with hosts; secure payments and account protection.

Barrier: Trust – how to know we are getting a good place? what if renters damage the property?
Solution: Used professional photos to inspire trust – to show that places weren’t dumps and had verified addresses; user profile verification, insurance options.

Barrier: Regulations – cities imposing bans on short term rentals.
Solution: positioned as a ‘platform to connect’ hosts with visitors, agreement putting host responsible for tax payments.

Barrier: Cultural Shift – how to get people comfortable with living in someone’s house?
Solution: intensive marketing campaigns branding as a local experience and ‘Belong Anywhere’.

Belong Anywhere
That’s how Airbnb is getting people to associate with the cultural shift

We saw how Airbnb started out, a robust strategy with a good market fit. It takes a lot more, though, to continue budding this innovative spirit. You need the right people.

The Present

If you have the right people, they will do great things for you. Right? Well, it is not so easy. What matters more than the people is your culture. You need to provide your organization with the right culture to grow and share.

Innovation, in itself never comes from one person or one idea. Innovation comes from the collision of diverse ideas. It is not thinking outside the box. It is more like thinking in multiple boxes at the same time.

This kind of mindset buds ideas. Ideas that are formed with knowledge. Knowledge from people working across divisions or departments. And the main reason why most “innovative” companies have open and shared workspaces. And that’s how Airbnb works.

Airbnb is not about booking or traveling, but about bringing people together and giving them a sense of belonging

Organizational Structure

Airbnb has created a strong culture of community and collaboration. They have setup an organizational structure to maximize networking within the organization. The managers exist not to lead but to facilitate information and act to remove obstacles for the team.

‘Ground Control’
An employee experience team that looks after the workplace environment, communication, and celebration.

Transnational Innovation

As they grew and moved into countries globally, Airbnb adopted the culture in those countries and adapted their strategy accordingly. Thus, they locally leveraged their innovation strategy and spread the learnings across all markets. That is truly transnational.

Airbnb has kept their information and knowledge free-flowing. They host community meetups to learn from their hosts and also use it as a repository to share this accrued knowledge.

Research Culture

‘Bulldozing‘ is when a feedback from researchers seem to decimate whatever product has been built. When researchers point out what is not working, it is definitely a good feedback, but how is it helping the product teams?

Instead, Airbnb makes the researchers work directly with product team throughout development and not just for testing. That is when it is easier to build something and test if it works. Researchers still tell what isn’t working and ‘bulldozing’, but now it is at the right time.

When it comes at the right time “bulldozing” is a good thing. It doesn’t flatten the building, it flattens the ground on which we build.
Airbnb’s Judd Antin, From the Ground Up

Airbnb has maintained its growth stats and quite recently surpassed Hilton in US consumer spending after expanding to boutique hotel rentals in 2018.

As Airbnb is prepping to go public with a much anticipated IPO, what would its future strategy look like?

The Future

Being a prime example of how to organize your culture and keep employees as close as family, Airbnb needs to increase the transparency between guests and hosts. At this moment, there are still a lot of issues with respect to broken promises and broken ~~hearts~~ apartments. Customers are facing problems from the likes of incorrect pictures to hiked prices.

If it hits an IPO, it will need to keep its trust ratings high and have a ton of tricks up its innovation pipeline.

Sustaining Innovation

With their extreme growth, Airbnb’s company size has also grown a lot. When new managers come in, they are not imbued into Airbnb’s culture and this adds a layer of bureaucratic barrier to innovation. The ‘managers’ need to be trained from grassroots to be ‘facilitators’ instead.

As with every new innovation, the first-mover advantage only lasts as long. Competitors (both hotels and new tech companies) are nibbling off market share from Airbnb. And that’s not even the problem! This competition is not based on quality. It is based on cost, which is never good for the market! (as it results into a race to the bottom)

Airbnb might soon have to come out with new product line offerings to maintain their lead. And for this they have to do what Airbnb does best; that’s not by striving for growth but constantly learning from the market. That’s when they will truly be able to “Belong Anywhere.“