Skip to content Skip to sidebar Skip to footer
Showing posts with the label Hadoop

Python Write To Hdfs File

What is the best way to create/write/update a file in remote HDFS from local python script? I am a… Read more Python Write To Hdfs File

Reading And Writing From Hive Tables With Spark After Aggregation

We have a hive warehouse, and wanted to use spark for various tasks (mainly classification). At tim… Read more Reading And Writing From Hive Tables With Spark After Aggregation

Hive Client For Python 3.x

is it possible to connect to hadoop and run hive queries using Python 3.x? I am using Python 3.4.1.… Read more Hive Client For Python 3.x

How To Run A Mrjob In A Local Hadoop Cluster With Hadoop Streaming?

I'm currently taking a Big Data Class, and one of my projects is to run my Mapper/Reducer on a … Read more How To Run A Mrjob In A Local Hadoop Cluster With Hadoop Streaming?

Hadoop Streaming: Where Are Application Logs?

My question is similar to : hadoop streaming: how to see application logs? (The link in the answer … Read more Hadoop Streaming: Where Are Application Logs?

Hive Parseexception In Drop Table Statement

I'm using python and pyodbc module in particular to execute Hive queries on Hadoop. The portion… Read more Hive Parseexception In Drop Table Statement

Unpickle File From Hdfs

I'm currently using Python 3 and would like to load a pickle file out of HDFS. from pywebhdfs.… Read more Unpickle File From Hdfs

How To Decode .lzo_deflat File?

does anyone have either a command-line or python library that can decode a .lzo_deflate file? Thes… Read more How To Decode .lzo_deflat File?