Job Description
Developed multiple hive scripts and redshift scripts for several workflows. Designed and implemented Customization of Keys, Values, Partitioners, Combiners, InputFormats and Record Readers in JAVA. Modify existing software to processing data using Spark (Pyspark, Scala, SparkSQL) and Spark Streaming, Integrated Spark Streaming with Kafka, Flume, Amazon Kinesis and Cassandra. Must be willing to travel to set up systems to various clients across the nation, perform feasibility studies, and evaluate user requirements.