All Questions
3,221
questions
-1
votes
0
answers
25
views
udf to transform a json string into multiple rows based on first level of nesting
I am trying to transform a df based on the first level nesting in the json string.
input dataframe
+------+------------------------------------+---------------------------------------------------------...
- 1,490
0
votes
0
answers
22
views
Flattening nested json with back slash in apache spark scala Dataframe
{
"messageBody": "{\"task\":{\"taskId\":\"c6d9fb0e-42ba-4a3e-bd39-f2a32a6958c1\",\"serializedTaskData\":\"{\\\"clientId\\\":\\\&...
0
votes
1
answer
44
views
Scala circe: encode List to json as flat string
The task is to encode:
List(1,2,3)
to the the following string:
[1,2,3]
By applying io.circe.asJson - function:
List(1,2,3).asJson
I receive the following output:
[
1,
2,
3
]
What am I ...
- 1,178
0
votes
0
answers
50
views
decoding json changing value and encoding in scala
I have this
request
.get(uri"$configurationEndpoint")
.header("Authorization", s"Bearer $token")
.send(backend)
...
0
votes
1
answer
41
views
Traverse through a JSON and add new fields in Scala
I have a JSON structure that looks like:
{
"A": {
"NestedArray": [
{
"AlreadyHere2": "x"
},
{
"AlreadyHere2": &...
- 35
1
vote
1
answer
64
views
Add nested fields to a JSON using Circe
Problem
How can I add fields to a Json object if the field is nested?
This is similar to this forum: Adding field to a JSON using Circe
but instead of:
{
ExistingField: {},
"Newfield" : {...
- 35
-2
votes
1
answer
56
views
Parse csv file with cell having comma and quotes using spark
My CSV file has more than 30 columns such that some of the columns represents json string like -
"{""name"": ""value1"",""name2"": &...
- 213
1
vote
0
answers
40
views
I have a Bigquery table with a field having JSON datatype, i have to use SCALA spark to load a string into this GCP json field
Found the same issue is raised for clarification which can be found here #880 and #882
And i tried the same things suggested.but it was not working.
My Error Message:-
2024-03-24T18:28:35.048Z ...
- 11
1
vote
1
answer
52
views
Convert Dataframe to nested Json in scala
I have a dataframe as below structure each level of json with *1, *2 like that.. and “->” shows the child node of a parent node
dataframe.show
id*1
name*1
ppu*1
type*1
toppings1->id2
toppings1-&...
- 17
0
votes
1
answer
48
views
How to convert a string json to scala dataframe?
I have a string json which i am trying to create a dataframe. using below code i can achieve but the environment i am building the jar, there its complaining about toDS. is there any other way t ...
- 822
1
vote
1
answer
59
views
Convert string column to JSON map of structs in scala
I'm trying to write a unit test that matches my data output, but struggling to create a sample dataframe of the right format.
The schema needs to look like this:
|-- ids: string (nullable = true)
...
- 23
1
vote
0
answers
43
views
json4s MappingException does not show up when throwing from my CustomSerializer
I have a Tag.scala file in my Play Framework app with TagKeys and TagValues defined. I want to throw a MappingException when the json is misformatted or the keys/values passed do not belong to TagKey ...
1
vote
1
answer
318
views
dump a json file in to form of binary file
I have a very big json file which is probably closely 1.5GB which is kinda of a format of
[
{
"a": 123,
"b": 143,
"c": "111",
"d": 0,...
- 51
0
votes
0
answers
52
views
Spark arraytype issue
In the kafka data that comes for me to process(json)
Yes raw data is converted from xml to json If the object in xml is single, it comes as a single ({}), but if the object comes more than once, it ...
- 13
0
votes
0
answers
61
views
Iterate Json object using Scala2
Newbie Scala user's here! I was trying to extract all elements in a JSON array one by one, so I was iterating this object. Since Scala doesn't have built-in JSON support, I used a third-party JSON ...
- 1,571