Skip to main content
Solved

Http connector and large JSON results


Freddy
Forum|alt.badge.img+16
  • Thinkwise Local Partner Brasil
  • 531 replies

I am working on an App and I want to incorporate the economic activities.. these are available via public API. (https://servicodados.ibge.gov.br/api/v2/cnae/subclasses

How do I process this? Because I know use a NVARCHAR(MAX) variable, but I don't think it can hold the whole result?  37.8 MB ..   

However it does process but only a part of all the economic activities.  Is there a way to save the JSON as a file and later on process it? 

Best answer by Mark Jongeling

Hi Freddy,

Nvarchar(max) is able to hold 2,1GB of data so that shouldn't be the problem.

The content returned by this API is the JSON which you can capture inside a process variable with nvarchar(max). In a process procedure you can then query the data in SQL using openjson() for example.

Alternatively, you may also write that JSON to a file using the Write file connector for later processing; depends on the Use case.

View original
Did this topic help you find an answer to your question?
This topic has been closed for replies.

2 replies

Mark Jongeling
Administrator
Forum|alt.badge.img+23
  • Administrator
  • 3945 replies
  • Answer
  • March 25, 2022

Hi Freddy,

Nvarchar(max) is able to hold 2,1GB of data so that shouldn't be the problem.

The content returned by this API is the JSON which you can capture inside a process variable with nvarchar(max). In a process procedure you can then query the data in SQL using openjson() for example.

Alternatively, you may also write that JSON to a file using the Write file connector for later processing; depends on the Use case.


Freddy
Forum|alt.badge.img+16
  • Author
  • Thinkwise Local Partner Brasil
  • 531 replies
  • March 26, 2022

I’ll have to troubleshoot, because I see more data in the DB but it only shows so many records and there is no cap defined in the settings.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings