Skip to content Skip to sidebar Skip to footer

How To Read Multiline Csv File In Pyspark

I'm using this tweets dataset with Pyspark in order to process it and get some trends according to the tweet's location. But I'm having a problem when I try to create the dataframe

Solution 1:

It looks like a multiline csv. Try doing

df = spark.read.csv("hashtag_donaldtrump.csv", header=True, multiLine=True)

Post a Comment for "How To Read Multiline Csv File In Pyspark"