Skip to main content

Process Data

After saving the token from the Authentication step, we can now use it to make another request for actually getting the collisions. To send another HTTP request, we have to select an HTTPCaller block and connect the output of the JSONExtractor mentioned here to the input of the HTTPCaller. The procedure for finding the blocks and settings is explained here.

HTTPCaller Settings​

  • Set the Request URL to https://integration.cloud.tes.ca/api/collision/v2/CollisionsByDate
  • Select POST in HTTP Method select box.
  • In the Headers section, you need to write Authorization in the Name column.
  • In the Value column, you need to first write Bearer and then click on the down-arrow icon and select Attribute Value and then token.

Th final view of the Header table:

  • And click on OK

Like before, we have to connect the output of the HTTPCaller to the input of a JSONExtractor to save the data we need from the response. The properties we are interested in, are results and errorMessages.

JSONExtractor Settings​

  • Set the Input Source to JSON Document.
  • Click on the down-arrow icon next to the Input Source. From the dropdown menu, choose Attribute Value and then select _response_body which is actually the response body of the HTTPCaller saved in _response_body attribute.

  • Now, we have to extract the JSON properties we want to save in the attributes.
  • In the Target Attribute column, write the results and its corresponding JSON Query value which is json[“results”].
  • Write another target attribute named errors and its corresponding JSON Query which is json[“errorMessages”].
  • Click on OK

With errorMessages, we can check whether we have an error or not. The results is the collision data. Next, we will select an AttributeKeeper from the Transformers to keep the results attribute in memory to lower the processing time later on.

AttributeKeeper Settings​

  • In General section, click on down-arrow icon and from the shown drop-down menu, choose Select Attributes.

  • From the list, choose results.

  • Click on OK

Next step is to use a JSONFragmenter and connect it to the output of the AttributeKeeper. JSONFragmentor is used to extract portions of Json into new FME features.

JSONFragmentor Settings​

  • In the Source section, Select JSON attribute from the Input Source select box.
  • Select results from the JSON Attribute select box.

  • In the General section, Click on the three dots icon next to the JSON Query input.

  • Write json[*], which returns the whole JSON object.

  • Select JSON for the Fragment as Format select box.
  • Select Yes for the Reject Features which Produce No Fragment.

  • In the Flattening section, Select Yes for the Flatten Query Results into Attributes select box.
  • Select Yes from the Recursively Faltten Objects/Arrays select box.
  • Click on the three-dots icon next to the Attributes to Expose to choose which attributes you want to save.
  • Here, we keep a set of properties in the attributes since we want to use them in the Chart and the Table.
  • Click on the first empty row, write dayOfWeek,accidentDate and accNumber.

  • And click on OK

The next step is to save the property dayOfWeek. As we saw a lot of times, we need to add an AttributeKeeper and connect the output of the JSONFragmenter to AttributeKeeper's input.

AttributeKeeper Settings​

  • In the General section, click on the icon with three dots, beside Attributes to Keep.

  • Choose dayOfWeek,accidentDate and accNumber. and click on OK.

At this point, we have to connect the output of the AttributeKeeper to two different blocks. One generates the Chart, and the other generates the Table.

For generating a chart

We have to add an Aggregator to aggregate our data. Depending on what type of report you want to generate, you can customize the Aggregator Settings. Like before, we have to connect the AttrubuteKeeper to Aggregator output.

Aggregator Settings​

  • Check the Group Processing checkbox. Click on the icon with three dots, and choose dayOfWeek. Select Process at End from the Group by Mode options.

  • In the General section, select the Mode to Attributes only.
  • Write dayOfWeekCount in the Count Attribute input. dayOfWeekCount will be the name of the attribute we are making by grouping the data.
  • In the Attribute Accumulation section, select Merge Incoming Attributes from the Accumulation Mode select box's options.
  • Select Merge Incoming Attributes from the Accumulation Mode select box's options in the Attribute Accumulation section.

For Generating a Table

  • Add a DateTimeConverter from the transformers to convert the date format of the accidentDate property. Connect the output of the AttributeKeepr to the input of the DateTimeConverter.
  • In the General section, select the accidentDate for the Datetime Attribute input.
  • Don't change the Input Format since the automatically detecting format is chosen by default.
  • Choose the Output Format you want from its select box's options.
  • Click on OK

After that, we need to add a Sorter to sort the accidentDate and connect it to the DateTimeConverter.

Sorter Settings​

  • In the Sort By section, select the accidentDate as the Attribute column's value, select Numeric for the Alpha/Num column and select Descending or Ascending for the Order column.
  • Click on OK

We are done with processing the data, now we have to add two HTMLReportGenerators and connect them to the blocks of Table data and Chart data.

You can follow the tutorial for creating the report here