top of page
Search
outmawhencudddingp
Feb 1, 20221 min read

Spark-read-specific-partitions

May 31, 2020 — Too small and too many partitions have certain disadvantages. ... users = spark.​read.load('/path/to/users').repartition('userId') joined1 ...












spark-read-specific-partitions


Oct 31, 2020 — I have often used PySpark to load CSV or JSON data that took a long ... To load certain columns of a partitioned collection you use fastparquet.. In old versions(say Spark 939c2ea5af





0 views0 comments

Recent Posts

See All

Comments

bottom of page