I am having trouble using PySpark. Indeed, when I try to use the global SparkContext, sc, in the notebook it says that it's not defined and in my jupyter terminal I have the following error :
I tried troubleshooting but any solution I found is telling me to modify the hdfs directory and create a directory to which I have read and write access. Even though it's the right solution I can't access the hdfs directory in home because I get a permission denied.
Access right problem with SparkContext
Hi,
I am having trouble using PySpark. Indeed, when I try to use the global SparkContext, sc, in the notebook it says that it's not defined and in my jupyter terminal I have the following error :
I tried troubleshooting but any solution I found is telling me to modify the hdfs directory and create a directory to which I have read and write access. Even though it's the right solution I can't access the hdfs directory in home because I get a permission denied.
Thank you
Note for information : this was resolved here https://discord.com/channels/812998447057469481/831773189847121931/837272284284715039
I have created a home directory for you in hdfs with your username and provided you with write access.
1
Hello, I have the same problem...
Note for information : this was resolved here https://discord.com/channels/812998447057469481/831773189847121931/837272284284715039
Add comment