Data Transform using pandas library in Python

“Over half of the time, analysts are trying to import/cleaning the data.”

— By numerous John/Jane Does of data analysts

Data these days can be flown in from various sources: web, database, local files, user input, etc. Analysts now often have to work with various format of data input, in order to make them compatible with each other for analysis. Though sometimes considered to be a data engineer’s work, data preparation is still an essential skills for all data analysts, especially those who work in small to medium size firms (as I am doing now).

I am going to introduce data reading/manipulation with pandas library in Python 3. I have recently worked extensively with pandas in Python 3 and started realized the powerful component in the library. In this post, I will the one I used most frequently, groupby() with pandas.

Continue reading


Merging Layers in ArcGIS

Esri is a company specialized in Geographic Information System tools. ArcGIS, ArcMap are two of the most commonly used and powerful tools among all the GIS tools provided. As more and more data in incorporating geo-coded information to provide details of certain events, GIS has become more and more popular among data analysts and business intelligent.

ArcGIS and ArcMap are available for download on The price for these tools are very expensive. If you belong to an educational institution, you may want to check with the related departments to see if such tools are provided free of charge within your institution. For businesses, contacting Esri getting a contract might be a better idea based on the scale of business. Here I am using ArcMap under free trial. You can sign up to use the software free of charge for a certain period of time.

Continue reading