Spark Read Json From Url Scala

We have seen here on how to parse JSON in Java using Gson and here on how to parse JSON in Groovy. Contribute to dblock/scala-parse-json development by creating an account on GitHub. spark_read_json: Read a JSON file into a Spark DataFrame in sparklyr: R Interface to Apache Spark rdrr. ATTENTION: The Scala code above was hard earned and is REALLY VALUABLE! Specifically, the “ val df2. Apache Spark is a fast and general engine for large-scale data processing. Sparks intention is to provide an alternative for Kotlin/Java developers that want to develop their web applications as expressive as possible and with minimal boilerplate. Requirement. I recently started investigating Apache Spark as a framework for data mining. ) Throughout this document, we will often refer to Scala/Java Datasets of Rows as DataFrames. I will introduce 2 ways, one is normal load using Put , and another way is to use Bulk Load API. It has support for reading csv, json, parquet natively. 进入Spark Shell. Once the data is loaded, however, figuring out how to access individual fields is not so straightforward. So searching StackOverflow and Google yields all kinds of responses that seem unnecessarily complicated. transfer method. Needing to read and write JSON data is a common big data task. Apache Spark SQL is able to work with JSON data through from_json(column: Column, schema: StructType) function. The JavaScript Object Notation format most widely utilized by Web applications for asynchronous frontend/backend communication. When starting the Spark shell, specify: the --packages option to download the MongoDB Spark Connector package. runQuery is a Scala function in Spark connector and not the Spark Standerd API. It's like JSON. URL paths use lower case, with dashes separating words. The (Scala) examples below of reading in, and writing out a JSON dataset was done is Spark 1. 2015): added spray-json-shapeless library Update (06. json with the following content. spark read json string java, spark read json string python, spark read json from s3, parsing json in spark-streaming, spark dataframe nested json,scala read json file,spark flatten json,spark. In the example below we create a mapping between a JSON object representing a Stripe charge and a Scala case class. 0, DataFrame is implemented as a special case of Dataset. Starting in the MEP 4. Scala and JSON. Now, since Spark 2. jsonFile - loads data from a directory of josn files where each line of the files is a json object. In this tutorial, we will learn what is Apache Parquet, It's advantages and how to read from and write Spark DataFrame to Parquet file format using Scala example. Reading JSON Nested Array in Spark DataFrames In a previous post on JSON data, I showed how to read nested JSON arrays with Spark DataFrames. Let us first understand the. nodejs连接mongodb的方法. Spark Streaming. Data lineage, or data tracking, is generally defined as a type of data lifecycle that includes data origins and data movement over time. So the requirement is to create a spark application which read CSV file in spark data frame using Scala. A very important ingredient here is scala. Scala URL FAQ: How do I download the contents of a URL to a String or file in Scala? I ran a few tests last night in the Scala REPL to see if I could think of different ways to download the contents of a URL to a String or file in Scala, and came up with a couple of different solutions, which I'll. By default, left unset. Spark SQL JSON with Python Overview. This article shows a sample code to load data into Hbase or MapRDB(M7) using Scala on Spark. Setting to path to our 'employee. json("people. 8 Direct Stream approach. Before we ingest JSON file using spark, it's important to understand JSON data structure. Spark对HDFS上json数据的操作非常方便,本文以两种方式进行简单介绍,分别为Spark Shell 和 编写Scala应用程序。 Spark Shell. toJson[T](T)(implicit writes: Writes[T]). It has support for reading csv, json, parquet natively. So how do you get the JSON representation of an. Reading Data From Oracle Database With Apache Spark In this quick tutorial, learn how to use Apache Spark to read and use the RDBMS directly without having to go into the HDFS and store it there. (ii)None of the options (iii)and Build in support to read data from various input formats like Hive, Avro, JSON, JDBC, Parquet, etc. DataFrames have become one of the most important features in Spark and made Spark SQL the most actively developed Spark component. read-json-files - Databricks. The MapR Database OJAI Connector for Apache Spark provides an API to save an Apache Spark RDD to a MapR Database JSON table. In fact, by default, the bytes generated by Python 3’s pickle cannot be read by a Python 2. jsonFile - loads data from a directory of josn files where each line of the files is a json object. Scala and Apache Spark might seem an unlikely medium for implementing an ETL process, but there are reasons for considering it as an alternative. Reading JSON from a File. Learn how to integrate Spark Structured Streaming and. JSON is one of the most common data interchange formats: a human-readable way of exchanging structured data that is ubiquitous throughout industry. It brings a new way of reading data apart from InputFormat API which was adopted from hadoop. Note that the file that is offered as a json file is not a typical JSON file. Structured data is nothing but tabular data which you can break down in rows and columns. Step1 : Create two different sample files - multiline and single line JSON file with above mentioned records (copy-paste). 0, we had only SparkContext and SQLContext, and also we would create StreamingContext (if using streaming). Scala URL FAQ: How do I download the contents of a URL to a String or file in Scala? I ran a few tests last night in the Scala REPL to see if I could think of different ways to download the contents of a URL to a String or file in Scala, and came up with a couple of different solutions, which I'll. Lets begin the tutorial and discuss about the SparkSQL and DataFrames Operations using Spark 1. Reading JSON from a File. springml:spark-sftp_2. A full program listing appears at the end of the article. Also, write RDD record…. You can directly input a URL into the editor and JSONLint will scrape it for JSON and parse it. If not specified, there is no limit. Search spark using scala jobs openings on YuvaJobs. Writing File into HDFS using spark scala ; 0 votes. Data source is an API for handling structured data in Spark. I was using json scala library to parse a json from a local drive in spark job :. By default, left unset. The schema of this DataFrame can be seen below. In the following example, we do just that and then print out the data we got:. Read JSON file to Dataset Spark Dataset is the latest API, after RDD and DataFrame, from Spark to work with data. Now we live. Dynamic cache which allows us to handle arbitrary method calls. In our example, we will be reading data from csv source. We’re going to parse a JSON file representing a Charge object from the popular Stripe payments API. json2csharp is joining forces with quicktype to offer new and improved features including JSON attributes, PascalCase properties, modern C# syntax (nullables, expression members), Dictionary detection, class deduplication, and more. JSON is a very common way to store data. This package can be added to Spark using the --packages command line option. We will show examples of JSON as input source to Spark SQL’s SQLContext. To run the example:. As an example, we will look at Durham police crime reports from the Dhrahm Open Data website. 0 and later, you can use S3 Select with Spark on Amazon EMR. Introduction In a previous article, I described how a data ingestion solution based on Kafka, Parquet, MongoDB and Spark Structured Streaming could have the following capabilities: Stream processing of data as it arrives. Spark – Read JSON file to RDD JSON has become one of the most common data format that is being exchanged between nodes in internet and applications. Reading resource files in Spark using Scala Often while coding up unit tests in Scala, I need to read from a file which is available in the resources folder. Part 1 focus is the "happy path" when using JSON with Spark SQL. json("path to file"). With Apache Spark you can easily read semi-structured files like JSON, CSV using standard library and XML files with spark-xml package. json" val people = spark. json file in HDFS. In this tutorial, I show how to run Spark batch jobs programmatically using the spark_submit script functionality on IBM Analytics for Apache Spark. • “Opening” a data source works pretty much the same way, no matter what. To start a Spark's interactive shell:. We will now work on JSON data. In the example below we create a mapping between a JSON object representing a Stripe charge and a Scala case class. After all, many Big Data solutions are ideally suited to the preparation of data for input into a relational database, and Scala is a well thought-out and. key or any of the methods outlined in the aws-sdk documentation Working with AWS credentials In order to work with the newer s3a:// protocol also set the values for spark. In both cases, you can start with the following. Then to transform the stream rdd into dataframe I recommend you look into flatMap, as you can map single column RDD into multiple columns after parsing the json content of each object. Spark Streaming is a Spark component that enables the processing of live streams of data. Home » Parsing key and values using Spark and Scala Parsing key and values using Spark and Scala My goal is to parse the following line, which is being read from Hive table and then i need to only parse the keys and store them into another new HIVE table. jsonRDD - loads data from an existing rdd where each element of the rdd is a string containing a json object. NET Documentation. Prepending s to any string literal allows the usage of variables directly in the string. We then define a Youngster DataFrame and add all the employees between the ages of 18 and 30. 4) you want to see the data in the DataFrame, then use this command. I'm trying to write a DataFrame to a MapR-DB JSON file. This chapter will explain how to use run SQL queries using SparkSQL. x application! JSON can be read by virtually any programming language – just scroll down on the official homepage to see implementations in all major and some minor languages. The Play JSON API provides implicit Writes for most basic types, such as Int, Double, String, and Boolean. That being said, I think the key to your solution is with org. Spark SQL provides an option for querying JSON data along with auto-capturing of JSON schemas for both. Loading and Saving Data in Spark. Read on to learn one more language and add more skills to your resume. This property is only specified when using an external Spark cluster; when Fusion is using its own standalone Spark cluster, this property isn't set. I'm reading a. scala Find file Copy path MaxGekk [SPARK-28141][SQL] Support special date values 051e691 Sep 22, 2019. By default Livy runs on port 8998 (which can be changed with the livy. Scala JSON FAQ: How can I parse JSON text or a JSON document with Scala? As I continue to plug away on my computer voice control application (), last night I started working with JSON, specifically the Lift-JSON library (part of the Lift Framework), which seems to be the preferred JSON library of the Scala community. Finally, let's map data read from people. Contribute to dblock/scala-parse-json development by creating an account on GitHub. It looks like SparkSession is part of the Spark's plan of unifying the APIs from Spark. It brings a new way of reading data apart from InputFormat API which was adopted from hadoop. In this article, we will check one of methods to connect Oracle database from Spark program. If you are just playing around with DataFrames you can use show method to print DataFrame to console. key or any of the methods outlined in the aws-sdk documentation Working with AWS credentials In order to work with the newer s3a. JSON is a very common way to store data. Json is the circe data type representing a JSON object. •In the Spark Scala shell (spark-shell) or pyspark, you have a SQLContext available automatically, as sqlContext. Using S3 Select with Spark to Improve Query Performance. There are a few things. If you know the schema in advance, use the version that specifies the schema to avoid the extra scan. Sample Input data can be the same as mentioned in the previous blog section 4. Aug 15, Here we are importing deriveDecoder which allow use to parse a JSON string based on Staff case class. The Charge object is complex and we only want to map it partially to a simple case class that fits the needs of our application. Allowing Spark to infer the schema is particularly useful, however, for scenarios when schemas change over time and fields are added or removed. After linking the spark library The next step is to create a Spark context object with the desired spark configuration that tells Apache Spark on how to access a cluster. From the community for the community | | |. Then to transform the stream rdd into dataframe I recommend you look into flatMap, as you can map single column RDD into multiple columns after parsing the json content of each object. Scala to JsValue conversion is performed by the utility method Json. We added dependencies for Spark SQL - necessary for Spark Structured Streaming - and for the Kafka connector. One reason why we love Apache Spark so much is the rich abstraction of its developer API to build complex data workflows and perform data analysis with minimal development effort. It is easy for humans to read and write. from_json (creates a JsonToStructs that) uses a JSON parser in FAILFAST parsing mode that simply fails early when a corrupted/malformed record is found (and hence does not support columnNameOfCorruptRecord JSON option). In the example below we create a mapping between a JSON object representing a Stripe charge and a Scala case class. Spark is an open source project for large scale distributed computations. spark read json string java, spark read json string python, spark read json from s3, parsing json in spark-streaming, spark dataframe nested json,scala read json file,spark flatten json,spark. 0 and above. I'm trying to dig into Scala a bit more, and one of the common exercises I do is to read in some JSON (being a common interchange format these days) and persist a simple Map back out to JSON. Sparks intention is to provide an alternative for Kotlin/Java developers that want to develop their web applications as expressive as possible and with minimal boilerplate. I have kept the content simple to get you started. Let us consider an example of employee records in a JSON file named employee. Scala to JsValue conversion is performed by the utility method Json. While XML is a first-class citizen in Scala, there's no "default" way to parse JSON. The json library was added to Python in version 2. Loading and Saving Data in Spark. Using S3 Select with Spark to Improve Query Performance. SQLContext(sc) Example. • "Opening" a data source works pretty much the same way, no matter what. JSON is a favorite among developers for serializing data. However you can try this. Abstract: The proposal of Spark DataSource API enables adaptability of various data sources to, Implement REST DataSource using Spark DataSource API. Initializing SparkSession A SparkSession can be used create DataFrame, register DataFrame as tables, execute SQL over tables, cache tables, and read parquet files. Play provides a very complete library for dealing with JSON objects, Play JSON. json file in HDFS. Data source is an API for handling structured data in Spark. This notebook uses Scala 2. ETL pipelines ingest data from a variety of sources and must handle incorrect, incomplete or inconsistent records and produce curated, consistent data for consumption by downstream applications. 07/12/2019; 7 minutes to read +4; In this article. You can also read from relational database tables via JDBC, as described in Using JDBC with Spark DataFrames. stringify() JSON. By default, left unset. Introduction In a previous article, I described how a data ingestion solution based on Kafka, Parquet, MongoDB and Spark Structured Streaming could have the following capabilities: Stream processing of data as it arrives. Apache Spark is a cluster computing system. port config option). scala apache-spark apache-kafka spark-streaming. ATTENTION: The Scala code above was hard earned and is REALLY VALUABLE! Specifically, the " val df2. Around 50 Scala library contributors, authors, and maintainers gathered and discussed various topics, such as the upcoming changes in Scala 3, improvements to the documentation and the website, the future of Scala tooling for both OSS developers and big companies, proposals to better collaborate online and welcome more contributors, etc. 0-bin-hadoop2. The set of possible orients is:. The guide is aimed at beginners and enables you to write simple codes in Apache Spark using Scala. This functionality depends on a converter of type Writes[T] which can convert a T to a JsValue. Let us explore the objectives of Running SQL Queries using Spark in the next section. How to save spark dataframe via Spark connector ⏩ Post By Niyaz Khafizov Intersystems Developer Community AI ️ API ️ Beginner ️ Machine Learning ️. Since Spark 2. 15): added circe library Some time ago I wrote a post on relational database access in Scala since I was looking for a library and there were many of them available, making it hard to make a choice. In single-line mode, a file can be split into many parts and read in parallel. •The DataFrame data source APIis consistent, across data formats. Stable and robust ETL pipelines are a critical component of the data infrastructure of modern enterprises. With Apache Spark you can easily read semi-structured files like JSON, CSV using standard library and XML files with spark-xml package. (iv)The top layer in the Spark SQL architecture. Livy is an open source REST interface for interacting with Apache Spark from anywhere. For more Apache Spark use-cases in general, I suggest you check out one of our previous posts. >>> df4 = spark. 11 validates your knowledge of the core components of the DataFrames API and confirms that you have a rudimentary understanding of the Spark Architecture. json() on either an RDD of String or a JSON file. Spark can automatically infer the schema of a JSON file loaded. Read on to learn one more language and add more skills to your resume. 11 for use with Scala 2. We then define a Youngster DataFrame and add all the employees between the ages of 18 and 30. I'm reading a. x application! JSON can be read by virtually any programming language – just scroll down on the official homepage to see implementations in all major and some minor languages. 10) Upon a successful installation, you will see an output (see Figure 2) followed by a Scala prompt (see Figure 3). ) however it does require you to specify the schema which is good practice for JSON anyways. So I have been lucky enough to work with Apache Spark for the last two years and in the countless projects I work on I find that there are usually many ways of doing the same thing, and sometimes…. Apache Spark, an open source data processing engine for batch processing, machine learning, data streaming and other types of analytics applications, is very significant example of Scala usage. Built for productivity. Aug 15, Here we are importing deriveDecoder which allow use to parse a JSON string based on Staff case class. springml:spark-sftp_2. Step1 : Create two different sample files - multiline and single line JSON file with above mentioned records (copy-paste). how to format scala's output from JSON to text file format Tag: json , scala , apache-spark , file-format , string-interpolation I am using Scala with Spark with below version. It was introduced in Spark 1. Spark builds upon Apache Hadoop, and allows a multitude of operations more than map-reduce. (JSON is also supported) should be as follows: keyColumn - key that can be used to perform de-duplication when reading. By default, left unset. Spark SQL allows you to write queries inside Spark programs, using. Before we ingest JSON file using spark, it's important to understand JSON data structure. It's like JSON. Requirement. Re: How to parse Json formatted Kafka message in spark streaming: Date: Thu, 05 Mar 2015 23:07:28 GMT: Hi, Helena, I think your new version only fits to the json that has very limited columns. This is a reference implementation. Spark builds upon Apache Hadoop, and allows a multitude of operations more than map-reduce. I used the json-smart cache library to do the actual parsing (it's really fast!) and wrote a wrapper in Scala to make the results nicer to use. When using the Spark Connector, it is impractical to use any form of authentication that would open a browser window to ask the user for credentials. toJavaRDD(). In Scala and Java, a DataFrame is represented by a Dataset of Rows. I will introduce 2 ways, one is normal load using Put , and another way is to use Bulk Load API. We broke this document into two pieces, because this second piece is considerably more complicated. sas7bdat) in parallel as data frame in Spark SQL. {a: '1'} is not valid JSON for a couple of reasons, from what I can tell: a needs to be a string ("a") and you need to use double quotes for "1". Published on 6 November 2015 , last updated on 6 June 2018. In fact, by default, the bytes generated by Python 3’s pickle cannot be read by a Python 2. In this quickstart, you use an Azure Resource Manager template to create an Azure Databricks workspace with an Apache Spark cluster. So I have been lucky enough to work with Apache Spark for the last two years and in the countless projects I work on I find that there are usually many ways of doing the same thing, and sometimes…. "Word Count" - This is the name of the application that you want to run. Search spark using scala jobs openings on YuvaJobs. > Dear all, > > > I'm trying to parse json formatted Kafka messages and then send back to cassandra. We are going to load a JSON input source to Spark SQL’s SQLContext. Introduction to Hadoop job. Parse JSON using Python. To use Structured Streaming with Kafka, your project must have a dependency on the org. Being asynchronous, actor-based, fast, lightweight, modular and testable it's a great way to connect your Scala applications to the world. That means we will be able to use JSON. There are a few variations to how this can be done, specifically if I am using the contents of the file as DataFrame in Spark. via builtin open function) or StringIO. It has support for reading csv, json, parquet natively. If the code uses sparklyr, You must specify the Spark master URL in spark_connect. In the example below we create a mapping between a JSON object representing a Stripe charge and a Scala case class. It brings a new way of reading data apart from InputFormat API which was adopted from hadoop. But if you. Hello, I'm a Spark beginner so go easy on me wonder if any Streaming gurus can help with this its driving me mad getting so confused with different Streaming formats etc :) I am reading from a Kafka topic using createDirectStream, and able to print out the json that I want to work on as per example below. I encourage you to read more about Spark Streaming from here in order to know more about its capabilities and do more advanced transformation on data for more insights in real time using it. SPARK-17232 Expecting same behavior after loading a dataframe with dots in column name Resolved SPARK-17341 Can't read Parquet data with fields containing periods ". Recently I have to pass JSON data to REST Service and did not have any simple Client handy. JSON (1) K-MUG (1). spark / sql / core / src / test / scala / org / apache / spark / sql / JsonFunctionsSuite. how to read multi-li… on spark read sequence file(csv o… Spack source code re… on Spark source code reading (spa… Spack source code re… on Spark source code reading (spa…. binary is more permissive than JSON because JSON includes field names, eg. This conversion can be done using SQLContext. In Scala and Java, a DataFrame is represented by a Dataset of Rows. After all, many Big Data solutions are ideally suited to the preparation of data for input into a relational database, and Scala is a well thought-out and. If you know the schema in advance, use the version that specifies the schema to avoid the extra scan. When “wholeFile” option is set to true (re: SPARK-18352), JSON is NOT splittable. For example, to include it when starting the spark shell: $ bin/spark-shell --packages com. In our application, we create a SparkSession and then create a DataFrame from a JSON file. The window would not necessarily appear on the client machine. using the read. ETL pipelines ingest data from a variety of sources and must handle incorrect, incomplete or inconsistent records and produce curated, consistent data for consumption by downstream applications. It can be created using a Reader object as demonstrated in this code or using a File corresponding to the JSON stream. Scala to JsValue conversion is performed by the utility method Json. Spark SQL can automatically infer the schema of a JSON dataset, and use it to load data into a DataFrame object. • “Opening” a data source works pretty much the same way, no matter what. a long that is too large will overflow an int), it is simpler and more reliable to use schemas with identical Parsing Canonical Form. The schema of this DataFrame can be seen below. In this chapter, we will walk you through using Spark Streaming to process live data streams. URL paths, URL query parameter names, and JSON field names are case sensitive. Spark can automatically infer the schema of a JSON file loaded. 15): added circe library Some time ago I wrote a post on relational database access in Scala since I was looking for a library and there were many of them available, making it hard to make a choice. I suggest you take the NetworkWordCount example as starting point. I also touched upon in brief about Groovy here. Play provides a very complete library for dealing with JSON objects, Play JSON. It is easy for machines to parse and generate. Open Source Apache Spark is fast becoming the de facto standard for Big Data processing and analytics. It shows your data side by side in a clear, editable treeview and in a code editor. Spark – Read JSON file to RDD JSON has become one of the most common data format that is being exchanged between nodes in internet and applications. For more Apache Spark use-cases in general, I suggest you check out one of our previous posts. SQLContext(sc) Example. I am seeing online videos where people are using some web app of cloudera to open these JSON files and then cutting n pasting the contents into. Scala URL FAQ: How do I download the contents of a URL to a String or file in Scala? I ran a few tests last night in the Scala REPL to see if I could think of different ways to download the contents of a URL to a String or file in Scala, and came up with a couple of different solutions, which I'll. I used the json-smart cache library to do the actual parsing (it's really fast!) and wrote a wrapper in Scala to make the results nicer to use. The following code examples show how to use org. Now that I am more familiar with the API, I can describe an easier way to access such data, using the explode() function. As you can see, the combination of Spark and Zeppelin is incredibly powerful. I have two problems: > 1. I wanted to parse the file and filter out few records and write output back as file. Aug 15, Here we are importing deriveDecoder which allow use to parse a JSON string based on Staff case class. MessagePack is an efficient binary serialization format. Basically, JSON (JavaScript Object Notation) is a lightweight data-interchange format. This property is only specified when using an external Spark cluster; when Fusion is using its own standalone Spark cluster, this property isn’t set. Then to transform the stream rdd into dataframe I recommend you look into flatMap, as you can map single column RDD into multiple columns after parsing the json content of each object. Loading and Saving Data in Spark. We can even use the %sql interpreter in Zeppelin to write a query without any Scala code and graph the results: Conclusion. Tips & Tricks. The Spark shell provides an easy and convenient way to prototype certain operations quickly,without having to develop a full program, packaging it and then deploying it. cmd script found in the bin folder to start Spark shell using Scala. It brings a new way of reading data apart from InputFormat API which was adopted from hadoop. We broke this document into two pieces, because this second piece is considerably more complicated. If you want to execute sql query in Python, you should use our Python connector but not Spark connector. This has a performance. Copy and paste, directly type, or input a URL in the editor above and let JSONLint tidy and validate your messy JSON code. I am new to Spark Streaming world. Structured data is nothing but tabular data which you can break down in rows and columns. •The DataFrame data source APIis consistent, across data formats. spray is an open-source toolkit for building REST/HTTP-based integration layers on top of Scala and Akka. parse() throws if the string passed to it has trailing commas. But created very simple Java program which read JSON data from file and sends it to REST service. Spark SQL can automatically infer the schema of a JSON dataset, and use it to load data into a DataFrame object. Spark Shell. Using with Spark shell. Update (18. To start a Spark's interactive shell:. Also, write RDD record…. This concludes our tutorial on Scala - How To Escape Characters and Create Multi-Line String and I hope you've found it useful! Stay in touch via Facebook and Twitter for upcoming tutorials! Don't forget to like and share this page :). {a: '1'} is not valid JSON for a couple of reasons, from what I can tell: a needs to be a string ("a") and you need to use double quotes for "1". toJavaRDD(). With Apache Spark you can easily read semi-structured files like JSON, CSV using standard library and XML files with spark-xml package. Prior to 2. Spark Streaming example on Consuming and Producing messages from/to Kafka in JSON format using Scala Language and from_json and to_json functions. play" % "play-json_2. Welcome to Livy. 0+ with python 3. If the code uses sparklyr, You must specify the Spark master URL in spark_connect. generate c# classes from a json string or url. You can read data from HDFS (hdfs://), S3 (s3a://), as well as the local file system (file://). simple is a simple Java library for JSON processing, read and write JSON data and full compliance with JSON specification (RFC4627) Warning This article is using the old JSON. 0 release there is an option to switch between micro-batching and experimental continuous streaming mode. 10 is similar in design to the 0. In the next series of blog posts, I will be discussing how to load and query different kind of structured data using data source API. If you want to execute sql query in Python, you should use our Python connector but not Spark connector. Put(For Hbase and MapRDB) This way is to use Put object to load data one by one.