Sandipan Ghosh – Medium

Sandipan Ghosh

Sandipan Ghosh

HOW TO PARSE XML DATA TO A SAPRK DATAFRAME

Purpose

Mar 14, 2022

HOW TO PARSE XML DATA TO A SAPRK DATAFRAME

Mar 14, 2022

Sandipan Ghosh

Data Streaming — AWS Kinesis

In my current organisation, we had implemented Kafka from end to end, from acquiring resources, installing Kafka, developing the producer…

Oct 16, 2021

Data Streaming — AWS Kinesis

Oct 16, 2021

Sandipan Ghosh

Cloud Computations — Quick data analysis with AWS Athena, Glue and Databricks spark

Throughout my carrier, I always had a situation that I had to fix failing production jobs. Most of the time, the debug involved analysis…

Oct 3, 2021

Cloud Computations — Quick data analysis with AWS Athena, Glue and Databricks spark

Oct 3, 2021

Sandipan Ghosh

Spark — Write single file per (hive) partitions

I have faced the problem of a small file in Hadoop too many times during the spark ETL process which is writing data to partitioned Hive…

Jun 25, 2021

Spark — Write single file per (hive) partitions

Jun 25, 2021

Sandipan Ghosh

How to Install Spark 3 on Windows 10

I have been using spark for a long time. It is an excellent, distributed computation framework. I use this regularly at work, and I also…

Mar 10, 2021

How to Install Spark 3 on Windows 10

Mar 10, 2021

Sandipan Ghosh

How to download really big data sets for big data testing

For a long time, I have been working with big data technologies, like MapReduce, Spark, Hive, and very recently I have started working on…

Aug 3, 2020

How to download really big data sets for big data testing

Aug 3, 2020

Sandipan Ghosh

Sandipan Ghosh

Bigdata solution architect & Lead data engg with experience in building data-intensive applications,tackling challenging architectural and scalability problems.

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams