NYSETL.Engines.E5.Producer (nys_etl v0.1.0) View Source
Link to this section Summary
Functions
handle_demand/2
is called when the Broadway dispatcher sees that consumers are free
to receive more messages.
Callback implementation for GenStage.handle_info/2
.
Callback implementation for GenStage.init/1
.
Link to this section Functions
handle_demand/2
is called when the Broadway dispatcher sees that consumers are free
to receive more messages.
This module iterates through counties in the current_county_list
until it runs out of
counties, at which point it goes to sleep. After being asleep idle_timeout_ms
, the :poll
timer will wake this producer up, at which point it will start over from the top of the
county list.
Special cases in the function heads below:
- backlog has message count greater than demand : A previous API request has put CommCare cases into a backlog, and we can drain the backlog without making another API request.
- is_nil(last_offset) : The last API request for the current county is the last page of records that have been updated recently. Move on to the next county.
Error handling:
- If an error is observed when making a CommCare API request, recurse into the
is_nil(last_offset)
special case so that we immediately move on to the next county.
Callback implementation for GenStage.handle_info/2
.
Callback implementation for GenStage.init/1
.