I have a maven project and i need import graphframe dependency to use spark grapx,this's my pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>demoGraphX</artifactId>
<version>1.0-SNAPSHOT</version>
<repositories>
<repository>
<id>SparkPackagesRepo</id>
<url>http://dl.bintray.com/spark-packages/maven</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-graphx_2.11</artifactId>
<version>2.2.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/graphframes/graphframes -->
<dependency>
<groupId>graphframes</groupId>
<artifactId>graphframes</artifactId>
<version>0.7.0-spark2.4-s_2.11</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.2.1</version>
</dependency>
</dependencies>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
</project>
But when i import graphframe dependency, my IDE return:
Cannot resolve org.apache.parquet:parquet-jackson:1.8.2
Cannot resolve com.twitter:chill_2.11:0.8.0
Cannot resolve org.apache.parquet:parquet-hadoop:1.8.2
Cannot resolve io.dropwizard.metrics:metrics-core:3.1.2
Cannot resolve org.glassfish.hk2:hk2-utils:2.4.0-b34
Cannot resolve com.univocity:univocity-parsers:2.2.1
Cannot resolve org.apache.directory.api:api-asn1-api:1.0.0-M20
Cannot resolve org.glassfish.hk2:osgi-resource-locator:1.0.1
Cannot resolve org.objenesis:objenesis:2.1
Cannot resolve commons-cli:commons-cli:1.2
Cannot resolve org.apache.hadoop:hadoop-yarn-common:2.6.5
Cannot resolve org.mortbay.jetty:jetty-util:6.1.26
Cannot resolve com.google.code.gson:gson:2.2.4
Cannot resolve com.chuusai:shapeless_2.11:2.3.2
etc...
I guest something wrong in my Maven. I use:
Intellij lastest
java 1.8.0_292
Apache Maven 3.6.3
Can you help me solve this problem. Thank you!
The bintray service was shutdown starting from 1st of May. (Press release)
So Apache spark community has provided new repo to host all spark packages. You can add/replace below code snippet in your code and things should work.
<repositories>
<repository>
<id>bintray</id>
<name>Bintray Repository</name>
<url>https://repos.spark-packages.org</url>
</repository>
</repositories>