Tag: bookbook pyspark

链接Spark与iPython Notebook

我在线学习了一些教程，但是他们不能在OS X El Capitan（10.11）上使用Spark 1.5.1 ， 基本上我已经运行这个命令下载apache-spark brew update brew install scala brew install apache-spark 更新.bash_profile # For a ipython notebook and pyspark integration if which pyspark > /dev/null; then export SPARK_HOME="/usr/local/Cellar/apache-spark/1.5.1/libexec/" export PYSPARK_SUBMIT_ARGS="–master local[2]" fi 跑 ipython profile create pyspark 创build了一个以这种方式configuration的启动文件~/.ipython/profile_pyspark/startup/00-pyspark-setup.py # Configure the necessary Spark environment import os import sys # Spark home […]