Insights from data? What to look for when search for insights?

This post is a revised version of presentation for Hack Santa Monica Meetup on November 17th, 2016. To learn more about Hack Santa Monica Meetup, please look here

Hack Santa Monica Meetup was initiated by non-profit group SixThirty Group (formerly known as Team SixThirty). For more information of SixThirty Group, please check here

Often in Business Intelligence field, people would say the first thing to do is to visualize data. However, one might be confused, among various visualization methods, what are the one visualization method that will reveal insights? And more exactly, what are called insights?

Insights, as it literally suggests, are things that are embedded in data and waiting for you to discover. That is somewhat a romantic way to put it, and the process of discovering one takes lots of patience and cautious. Different people might be looking at different things. Accountants may look at ledgers and balance sheets, whereas economists may look at annual labor data or stock market data.

Here I use data from Santa Monica Open Data Portal, which is a data set describing Santa Monica’s fire report records.

Key Takeaways

  • Look for pattern
  • Look for anomaly
  • Knowing the context

Continue reading

Data Visualization: ggplot2 in R Basics

No matter what kind of data work you are doing, at the end of every task, people, your audiences will always want to see some graphs. Even people with most sophisticated data/programming skills want to see some simple illustration that immediately deliver the message. In this case, data visualization is very important.

I am using R as my primary language when analyzing data. R is a very powerful language and for most people without programming experience, it is easier than most other language. And this conclusion is based on learning Python and C++ by myself.

R has a very powerful package called ggplot2. GG stands for Graphic Grammar, an idea developed by Leland Wilkinson. There is a book that he wrote about this package and the idea behind it. If you are interested in learning more complex ideas about this graphic grammar, you can find this book online.

Here I will just do a simple illustration of what a common ggplot code looks like and how each part works.

Continue reading