NYSETL.Engines.E5.Producer (nys_etl v0.1.0) View Source

Link to this section Summary

Functions

handle_demand/2 is called when the Broadway dispatcher sees that consumers are free to receive more messages.

Callback implementation for GenStage.handle_info/2.

Callback implementation for GenStage.init/1.

Link to this section Functions

Link to this function

handle_demand(demand, state)

View Source

handle_demand/2 is called when the Broadway dispatcher sees that consumers are free to receive more messages.

This module iterates through counties in the current_county_list until it runs out of counties, at which point it goes to sleep. After being asleep idle_timeout_ms, the :poll timer will wake this producer up, at which point it will start over from the top of the county list.

Special cases in the function heads below:

  • backlog has message count greater than demand : A previous API request has put CommCare cases into a backlog, and we can drain the backlog without making another API request.
  • is_nil(last_offset) : The last API request for the current county is the last page of records that have been updated recently. Move on to the next county.

Error handling:

  • If an error is observed when making a CommCare API request, recurse into the is_nil(last_offset) special case so that we immediately move on to the next county.
Link to this function

handle_info(atom, state)

View Source

Callback implementation for GenStage.handle_info/2.

Callback implementation for GenStage.init/1.