IllegalArgumentException: "Error while instantiating 'org.apache.spark.sql.hive.HiveSessionStateBuilder':"


Error: IllegalArgumentException: "Error while instantiating 'org.apache.spark.sql.hive.HiveSessionStateBuilder':"


Hive session error are pretty common in Spark and I was able to suppress most of error by below solution.

The most common error in Spark session user face is related to hive session. Some error says hive can not load some says can not initiate.

I have shared one of issues here:





The quick fix is

Step 1: Go to spark folder, (It worked in multiple versions for me)


Step 2: Search for shell.py file (i found 2 instances of file).



  • C:\sparkinstall\spark-2.2.0-bin-hadoop2.7\python\pyspark\python\pyspark
  • C:\sparkinstall\spark-2.2.0-bin-hadoop2.7\python\pyspark






Step 3. Create backup of both files to do rollback in case you goof up.




Step 4. Search for ".enableHiveSupport()\" in file






Step 6. I will modify both files by removing code line ".enableHiveSupport()\"




Step 5: If your spark instance is running then stop and start your spark engine



Now I am able to run my spark session

No comments:
Write comments

Please do not enter spam links

Meet US

Services

More Services