[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[#DYF-938766]: Problem in TIGGE protocol for missing data

Manuel,

[You might receive this email twice]

> We have discovered a problem in the TIGGE protocol
> (http://tigge.ecmwf.int/ldm_protocol.html) while exchanging data with
> CMA. At the moment, CMA sends very few products (1920), which
> exacerbates this problem:
> 
> Since we have different REQUEST lines (=parallel transfers) for data and
> for 'protocol' messages, e.g.:
> REQUEST ANY "z_tigge_c_babj.*\.grib:.*(00|20|40|60|80)$"
> tigge-ldm.cma.gov.cn
> REQUEST ANY "z_tigge_c_babj.*\.grib:.*(01|21|41|61|81)$"
> tigge-ldm.cma.gov.cn
> ...
> REQUEST ANY "z_tigge_c_babj.*\.(manifest|done)$"   tigge-ldm.cma.gov.cn
> 
> It seems that after CMA pqinserts data and .done, the file .done arrives
> much faster. This triggers the process to check missing fields at our
> end. We send the file .missing asking for non-received products to be
> resend. Then CMA resends the data files and the .done, which again
> arrives much faster than the data. This ping-pong works while we receive
> data in between 2 .missing notifications. But if we don't receive any
> data, the list of missing fields would not be sent since it contains the
> same list of products, ie, it has the same MD5.
> We have exercised this problem and this is the list of .done
> notifications received (note that we had to force resending a .missing):

While I'm not completely sure I understand the details of what your're
doing, I think you can solve your problem by adding a timestamp line
to the data-product that names the "missing" data.  This would make its
MD5 signature unique.

Is this possible?


Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: DYF-938766
Department: Support IDD TIGGE
Priority: High
Status: On Hold