pyiem.nws.products package¶

Submodules¶

pyiem.nws.products.cf6 module¶

Parser for the CF6 Product.

class pyiem.nws.products.cf6.CF6Product(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a CF6 Product

parser()¶: Do the parsing we need to do!

sql(cursor)¶: Send the data to the database.

pyiem.nws.products.cf6.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Provide back CF6 objects based on the parsing of this text

pyiem.nws.products.cli module¶

Parser and object storage of information within NWS CLI Product.

class pyiem.nws.products.cli.CLIProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a CLI Daily Climate Report Product

compute_diction(text)¶: Try to determine what we have for a format

find_sections()¶

Some trickery to figure out if we have multiple reports

Returns:: list of text sections

get_jabbers(uri, _=None)¶: Override the jabber message formatter

parse_data(section)¶: Actually do the parsing of this silly format

sql(cursor)¶: Do the database update!

pyiem.nws.products.cli.convert_key(text)¶: Convert a key value to something we store

pyiem.nws.products.cli.get_number(text)¶: Convert a string into a number, preferable a float!

pyiem.nws.products.cli.get_number_year(text)¶: Ensure we get a year that makes sense.

pyiem.nws.products.cli.make_tokens(regime, line)¶: Turn a line into tokens based on a regime

pyiem.nws.products.cli.parse_headline(section)¶: Figure out when this product is valid for

pyiem.nws.products.cli.parse_precipitation(regime, lines, data)¶: Parse the precipitation data

pyiem.nws.products.cli.parse_sky_coverage(lines, data)¶: Turn section into data.

pyiem.nws.products.cli.parse_snowfall(regime, lines, data)¶: Parse the snowfall data

pyiem.nws.products.cli.parse_temperature(prod, regime, lines, data)¶: Here we parse a temperature section

pyiem.nws.products.cli.parse_wind(lines, data)¶: Parse any wind information.

pyiem.nws.products.cli.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Parse CLI Text Products.

Parameters:: nwsli_provider (dict) – This dictionary provider in the form of the pyiem.network.Table object should contain additional attributes of access_station and access_network to map back to IEMAccess.

pyiem.nws.products.cli.sql_data(prod, cursor, data)¶: Do an individual data entry.

pyiem.nws.products.cli.trace_r(val)¶: Convert our value back into meaningful string

pyiem.nws.products.cli.update_iemaccess(txn, entry)¶: Update the IEM Access Database.

pyiem.nws.products.cwa module¶

Center Weather Advisories (CWA)

class pyiem.nws.products.cwa.CWAProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a Center Weather Advsiory (CWA) product.

get_jabbers(_uri, _uri2=None)¶: Return the Jabber for this sigmet

sql(txn)¶: Do SQL related stuff that is required

pyiem.nws.products.cwa.go2lonlat(lon0, lat0, direction, displacement)¶: http://stackoverflow.com/questions/7222382

pyiem.nws.products.cwa.parse_polygon(prod: TextProduct, line: str) → Tuple[Polygon, str]¶: Figure out what the polygon is!

pyiem.nws.products.cwa.parse_product(prod: TextProduct) → CWAModel¶: Do the parsing we need for the data model.

pyiem.nws.products.cwa.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.dsm module¶

Parser of the Daily Summary Message (DSM).

class pyiem.nws.products.dsm.DSMCollective(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: WMOProduct

A collective representing a NOAAPort Text Product with many DSMs.

sql(txn)¶: Do databasing.

tzlocalize(tzprovider)¶: Localize our currently stored timestamps.

class pyiem.nws.products.dsm.DSMProduct(groupdict)¶

Bases: object

Represents a single DSM.

compute_times(utcnow)¶: Figure out when this DSM is valid for.

sql(txn)¶: Persist to database given the transaction object.

tzlocalize(tzinfo)¶: Localize the timestamps, tricky.

pyiem.nws.products.dsm.compute_time(date, timestamp)¶: Make a valid timestamp.

pyiem.nws.products.dsm.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Provide back DSM objects based on the parsing of this text

pyiem.nws.products.dsm.process(text)¶: Emit DSMProduct object for what we can parse.

pyiem.nws.products.ero module¶

Eh, I am able to parse the SPC PTS now, so why not add more pain.

Weather Prediction Center Excessive Rainfall Outlook.

class pyiem.nws.products.ero.ERO(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

A class representing the polygons and metadata in WPC ERO Product

draw_outlooks()¶: For debugging, draw the outlooks on a simple map for inspection!

find_issue_expire()¶: Determine the period this product is valid for

find_outlooks()¶: Find the outlook sections within the text product!

get_jabbers(uri, _uri2=None)¶: Wordsmith the Jabber/Twitter Messaging

get_outlook(category, threshold, day)¶: Get an outlook by category and threshold

get_outlookcollection(day)¶: Returns the OutlookCollection for a given day

sql(txn)¶

Do database work

Parameters:: txn (psycopg.cursor) – database cursor

pyiem.nws.products.ero.compute_loc(lon, lat, dist, bearing)¶: Estimate a lat/lon

pyiem.nws.products.ero.compute_wfos(outlook_collections)¶: Compute the WFOs affected for each outlook in collection.

pyiem.nws.products.ero.init_days(prod)¶: Figure out which days this product should have based on the AFOS.

pyiem.nws.products.ero.jabber_messages(valid, outlook_collections) → list¶

Generate a list of Jabber messages to send.

Parameters:

valid (datetime) – the valid time of the outlook
outlook_collections (dict) – a list of OutlookCollection objects

Returns:

a list of Jabber messages [(plain, html, xtradict), …]

Return type:

list

pyiem.nws.products.ero.meat2segment(meat)¶: Convert into a list of points.

pyiem.nws.products.ero.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Parse this text!

pyiem.nws.products.ero.sid_rectify(sid)¶: Ensure it matches our nomenclature.

pyiem.nws.products.fd module¶

Parser for the FD (Temp Wind Aloft Forecasts).

class pyiem.nws.products.fd.FDProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a FD Product

parser()¶: Do the parsing we need to do!

sql(cursor)¶: Send the data to the database.

pyiem.nws.products.fd.compute_time(valid, tokens)¶: Figure out the timestamp of interest here.

pyiem.nws.products.fd.make4(station, afos)¶: Make this 3 character station, 4!

pyiem.nws.products.fd.parse_encoding(text)¶: Convert the encoded text into drct, sknt, tmpc.

pyiem.nws.products.fd.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Provide back FD objects based on the parsing of this text

pyiem.nws.products.ffg module¶

Parsing of Flash Flood Guidances

NWS Discontinued 30 Sep 2018 https://www.weather.gov/media/notification/pdfs/pns18-13disc_county_ffg.pdf

class pyiem.nws.products.ffg.FFGProduct(text, utcnow=None)¶

Bases: TextProduct

Class representing a FFG Product

do_parsing()¶: Process this file and save data

sql(txn)¶

Do the necessary database work

Parameters:: (psycopg.transaction) – a database transaction

pyiem.nws.products.ffg.parser(text, utcnow=None)¶

parser of raw SPC SAW Text

Parameters:

text (str) – the raw text to parse
utcnow (datetime) – the current datetime with timezone set!

Returns:

SAWProduct instance

pyiem.nws.products.ffg.safe(val)¶: Safe conversion to float

pyiem.nws.products.gairmet module¶

Aviation Weather Center Graphical-AIRMET G-AIRMET.

Break-up the XML G-AIRMET into atomic pieces.

class pyiem.nws.products.gairmet.GAIRMET(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Class for parsing the G-AIRMET product

parsing()¶: Attempt to parse out what we have found

sql(cursor)¶: Persist this information to the database

pyiem.nws.products.gairmet.parseUTC(s)¶: Parse an ISO-ish string into UTC timestamp

pyiem.nws.products.gairmet.parser(buf, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Parse a G-AIRMET product

Parameters:: buf (str) – What we want to parse

pyiem.nws.products.gairmet.prettify(elem)¶: Return a pretty-printed XML string for the Element.

pyiem.nws.products.gairmet.process_airmet(prod, airmet)¶: Process an AIRMET element

pyiem.nws.products.hml module¶

NWS Hydrological Markup Language

Attempt to break up the HML product into atomic data

class pyiem.nws.products.hml.HML(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Class for parsing and representing Space Wx Products

do_sql_forecast(cursor, _hml)¶: Process the forecast portion of the dataset

do_sql_observed(cursor, _hml)¶: Process the observed portion of the dataset

parsing()¶: Attempt to parse out what we have found

sql(cursor)¶: Persist this information to the database

class pyiem.nws.products.hml.HMLData¶

Bases: object

Our data object.

pyiem.nws.products.hml.no999(val)¶: No negative -999 or -9999 please.

pyiem.nws.products.hml.parseUTC(s)¶: Parse an ISO-ish string into UTC timestamp

pyiem.nws.products.hml.parse_xml(token)¶: Attempt to parse the XML into something useful

pyiem.nws.products.hml.parser(buf, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Parse a HML NOAAPort product

This may have multiple xml documents inside.

Parameters:: buf (str) – What we want to parse

pyiem.nws.products.hwo module¶

Hazardous Weather Outlook.

class pyiem.nws.products.hwo.HWOProduct(text, utcnow=None, ugc_provider: UGCProvider | dict | None = None, nwsli_provider=None, parse_segments=True)¶

Bases: TextProduct

Represents a HWO

get_channels()¶: overridden TextProduct#get_channels

pyiem.nws.products.hwo.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.lsr module¶

NWS Local Storm Report (LSR) Parsing.

class pyiem.nws.products.lsr.LSRProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a text product of the LSR variety

get_jabbers(uri, _uri2=None)¶: return a text and html variant for Jabber stuff

get_temporal_domain()¶: Return the min and max timestamps of lsrs

get_url(baseuri)¶: Get the URL of this product

is_summary()¶: Returns is this LSR is a summary or not

exception pyiem.nws.products.lsr.LSRProductException¶

Bases: TextProductException

Something we can raise when bad things happen!

pyiem.nws.products.lsr.parse_lsr(prod, text)¶: Emit a LSR object based on this text! 0914 PM HAIL SHAW 33.60N 90.77W 04/29/2005 1.00 INCH BOLIVAR MS EMERGENCY MNGR

pyiem.nws.products.lsr.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function that actually converts the raw text and emits an LSRProduct instance or returns an exception

pyiem.nws.products.mcd module¶

Supports parsing of Storm Prediction Center’s MCD and parsing of Weather Prediction Center’s MPD

class pyiem.nws.products.mcd.MCDProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a Storm Prediction Center Mesoscale Convective Discussion

database_save(txn)¶: Save this product to the database

find_valid_times()¶: Figure out when this product is valid for

find_watch_probability()¶: Find the probability of watch issuance for SPC MCD

get_jabbers(uri, _uri2=None)¶

Return plain text and html variants for a Jabber msg

Parameters:

uri (str) – URL number one needed for constructing the URI
_uri2 (str) – not used, but needed for the over-ride

Returns:

List of lists, plain text, html text, xtra dict

Return type:

(list)

get_url()¶: Return the URL for SPC’s website

parse_areas_affected()¶: Return the areas affected

parse_concerning()¶: Figure out the concerning text, if it exists.

parse_correction() → bool¶: Look for the correction flag.

parse_discussion_num()¶: Figure out what discussion number this is

parse_geometry()¶: Find the polygon that’s in this MCD product

tweet()¶: Return twitter message

pyiem.nws.products.mcd.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.metarcollect module¶

Encapsulates a text product holding METARs.

class pyiem.nws.products.metarcollect.METARCollective(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

A TextProduct containing METAR information

get_jabbers(uri, _uri2=None)¶: Make this into jabber messages

split_and_parse()¶: Create METAR objects as we find products in the text

pyiem.nws.products.metarcollect.normalize_temp(val)¶: When temperatures are close to an int, return that int!

pyiem.nws.products.metarcollect.normid(station_id: str) → str¶: Normalize a station identifer.

pyiem.nws.products.metarcollect.over_wind_threshold(mtr: Metar) → bool¶: Is this METAR over the wind threshold for alerting

pyiem.nws.products.metarcollect.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.metarcollect.sanitize(text)¶: Clean our text string with METAR data

pyiem.nws.products.metarcollect.to_iemaccess(txn, mtr: Metar, iemid: int, tzname: str, force_current_log=False, skip_current=False)¶

Persist parsed data to IEMAccess Database.

Parameters:

txn (psycopg.cursor) – database cursor / transaction
mtr (Metar) – Metar instance
iemid – The iem station identifier
tzname (str) – Local timezone of station.
force_current_log (boolean) – should this ob always go to current_log
skip_current (boolean) – should this ob always skip current table

pyiem.nws.products.metarcollect.to_metar(textprod, text) → Metar¶: Create a METAR object, if possible

pyiem.nws.products.metarcollect.trace(pobj)¶: Convert this precip object to a numeric value

pyiem.nws.products.metarcollect.wind_logic(iem, mtr: Metar)¶: Hairy logic for now we handle winds.

pyiem.nws.products.metarcollect.wind_message(mtr: Metar) → Tuple[str, int]¶: Convert this into a Jabber style message

pyiem.nws.products.mos module¶

Supports parsing of Textual Model Output Statistics files

class pyiem.nws.products.mos.MOSProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: WMOProduct

Represents a Model Output Statistics file

parse_data()¶: Parse out our data!

sql(txn)¶

Persist our data to the database

Parameters:: txn – Database cursor
Returns:: int number of inserts made to the database

pyiem.nws.products.mos.make_null(val)¶: Hmmm, perhaps we should set 999 as null too?

pyiem.nws.products.mos.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.mos.section_parser(sect)¶: Parse this section of text

pyiem.nws.products.nhc module¶

Handle some of the fun things that come from the Hurricane Center

class pyiem.nws.products.nhc.NHCProduct(text, utcnow=None, ugc_provider: UGCProvider | dict | None = None, nwsli_provider=None, parse_segments=True)¶

Bases: TextProduct

Represents a NHC

get_jabbers(uri, _uri2=None)¶: Get the jabber variant of this message

pyiem.nws.products.nhc.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.nldn module¶

http://www.unidata.ucar.edu/data/lightning/nldn.html

class pyiem.nws.products.nldn.NLDNProduct¶

Bases: object

Simple class representing obs found in a NLDNProduct.

sql(cursor)¶: Persist data to the database

pyiem.nws.products.nldn.parser(buf)¶: NLDN (unencrypted product)

pyiem.nws.products.pirep module¶

Pilot Reports (PIREP)

This module attempts to process and store atomic data from PIREPs. These are encoded products that look like so:

UBUS01 KMSC 221700 EAU UA /OV EAU360030/TM 1715/FL350/TP B737/TB CONT LGT-MOD CHOP = EHY UA /OV MBW253036 /TM 1729 /FL105 /TP C206 /SK FEW250 /TA M06 /TB NEG /RM SMTH=

Unfortunately, there is not much documentation of this format and the feed of this data contains a bunch of formatting errors.

class pyiem.nws.products.pirep.Pirep(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Class for parsing and representing PIREPs found in NWS text products.

assign_cwsu(txn)¶: Use this transaction object to assign CWSUs for the pireps

compute_loc(loc, dist, bearing)¶: Figure out the lon/lat for this location

compute_pirep_valid(hour, minute)¶: Based on what utcnow is set to, compute when this is valid

get_jabbers(_uri, _uri2=None)¶: get jabber messages

parse_reports()¶: Actually do the parsing of the product that generates the reports stored within the self.reports list

process_pirep(report)¶: Convert this report text into an actual PIREP object

sql(txn)¶: Save the reports to the database via the transaction

pyiem.nws.products.pirep.parser(buf, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: A parser implementation

pyiem.nws.products.saw module¶

Parsing of Storm Prediction Center SAW Product

This does not process the legacy SAW products that did not have LAT…LON

class pyiem.nws.products.saw.SAWProduct(text, utcnow=None)¶

Bases: TextProduct

Class representing a SAW Product

CANCELS = 1¶

ISSUES = 0¶

SEVERE_THUNDERSTORM = 1¶

TORNADO = 0¶

compute_wfos(_txn=None)¶: Figure out who is impacted by this watch

find_action()¶

Figure out if this is an issuance or cancells statement

Returns:: either ISSUES or CANCELS
Return type:: (int)

find_polygon()¶

Search out the text for the LAT…LON polygon

Returns:: Well Known Text (WKT) representation
Return type:: (str)

find_replaces() → int | None¶: Figure out what this watch replaces.

find_time()¶

Find the start and end valid time of this watch

Returns:: representing the time of this watch
Return type:: (datetime, datetime)

find_ww_num()¶

Find the Weather Watch Number

Returns:: The Weather Watch Number
Return type:: (int)

find_ww_type()¶

Find the Weather Watch Type

Returns:: The Weather Watch Type
Return type:: (int)

get_jabbers(uri, _uri2=None, **kwargs)¶

Generate the jabber messages for this product.

NOTE: In the past, the messages generated here have tripped twitter’s spam logic, so we are careful to craft unique messages

NOTE: Since interesting watch information comes within three products from SPC, there is some magic in pyWWA/watch_parser.py that awaits the arrival of all three.

Parameters:

uri (str) – link to IEM Watch Overview page.
_uri2 (str) – unused in this context.
wwpprod (WWPProduct) – the WWPProduct object.
selprod (SELProduct) – the SELProduct object.

is_test()¶

Is this a test watch?

Returns:: boolean if this SAW is a test or not

sql(txn)¶

Do the necessary database work

Parameters:: (psycopg.transaction) – a database transaction

pyiem.nws.products.saw.parser(text, utcnow=None, _ugc_provider=None, _nwsli_provider=None)¶

parser of raw SPC SAW Text

Parameters:

text (str) – the raw text to parse
utcnow (datetime) – the current datetime with timezone set!

Returns:

SAWProduct instance

pyiem.nws.products.scp module¶

Parsing NESDIS Satellite Cloud Product (SCP).

class pyiem.nws.products.scp.SCP(station, valid, mid, high, cldtop1, cldtop2, eca)¶

Bases: tuple

cldtop1¶: Alias for field number 4

cldtop2¶: Alias for field number 5

eca¶: Alias for field number 6

high¶: Alias for field number 3

mid¶: Alias for field number 2

station¶: Alias for field number 0

valid¶: Alias for field number 1

class pyiem.nws.products.scp.SCPProduct(text, utcnow=None)¶

Bases: TextProduct

Class representing a SCP Product

sql(txn)¶

Do the necessary database work

Parameters:: (psycopg.transaction) – a database transaction

pyiem.nws.products.scp.parser(text, utcnow=None)¶

parser of raw SCP Text.

Parameters:

text (str) – the raw text to parse
utcnow (datetime) – the current datetime with timezone set!

Returns:

SCPProduct instance

pyiem.nws.products.sel module¶

Parsing of Storm Prediction Center SEL Product.

class pyiem.nws.products.sel.SELProduct(text, utcnow=None)¶

Bases: TextProduct

Class representing a SEL Product

is_test()¶: Is this a test product?

sql(txn)¶

Do the necessary database work

Parameters:: (psycopg.transaction) – a database transaction

pyiem.nws.products.sel.parser(text, utcnow=None, _ugc_provider=None, _nwsli_provider=None)¶

Parse SPC SEL Product.

Parameters:

text (str) – the raw text to parse
utcnow (datetime) – the current datetime with timezone set!

Returns:

WWPProduct instance

pyiem.nws.products.shef module¶

Standard Hydrological Exchange Format (SHEF).

https://www.weather.gov/media/mdl/SHEF_CodeManual_5July2012.pdf

Formats¶

.A - single station, multiple parameter .B - multiple station, multiple parameter, header driven .E - single station, single parameter, evenly spaced time series

Implementation Notes¶

The IEM uses a nomenclature of 0.0001 float value to represent Trace values,

not the 0.001 that SHEF does. - When DH is blank or DHM, this generates a SHEFElement with a None valid attribute. This is ambiguous behavior to consider further.

TODO List¶

4.4.4 DIE special end-of-month specifier

5.1.4 how to handle repeated data

Handle when R is being specified in the AR BR ER

5.1.6 revision of a missing value

5.2.1 DR codes, DRE end of month

Table 9a D codes

4.4.1 ugly time logic and DN support

Handle UQ variable SSS.SDDD

Handle UR (tens of degrees)

class pyiem.nws.products.shef.SHEFProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

A single text product containing SHEF encoded data.

pyiem.nws.products.shef.clean_b_headerline(text)¶: Account for invalid encoding. SIGH.

pyiem.nws.products.shef.compute_num_value(element) → bool¶: Attempt to make this into a float.

pyiem.nws.products.shef.datetime24(dt, replacements)¶: Handle the junkiness that is a 24 hour.

pyiem.nws.products.shef.make_date(text, now=None)¶: Make the text date unambiguous.

pyiem.nws.products.shef.parse_A(prod) → int¶: Parse A format SHEF data.

pyiem.nws.products.shef.parse_B(prod) → int¶: Parse B format SHEF data.

pyiem.nws.products.shef.parse_E(prod) → int¶: Parse E format SHEF data.

pyiem.nws.products.shef.parse_d_wrapper(func)¶: Catch exceptions.

pyiem.nws.products.shef.parse_dc(text, basevalid)¶: Convert the DC element into a timestamp.

pyiem.nws.products.shef.parse_dd(text, valid)¶: Inner.

pyiem.nws.products.shef.parse_dh(text, valid)¶: Inner.

pyiem.nws.products.shef.parse_dm(text, valid)¶: Inner.

pyiem.nws.products.shef.parse_dt(text, valid)¶: Inner.

pyiem.nws.products.shef.parse_dy(text, basevalid)¶: Convert the DY element into a timestamp.

pyiem.nws.products.shef.parse_station_valid(text, utcnow)¶

Parse the first token found in a SHEF observation.

Parameters:

text (str) – the first part of the string
utcnow (datetime) – The default time.

Return type:

str, datetime, datetime, list

pyiem.nws.products.shef.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Parser.

pyiem.nws.products.shef.process_di(text)¶: Convert a DI value into an interval.

pyiem.nws.products.shef.process_message_a(prod, message) → List[SHEFElement]¶: Convert the message into an object.

pyiem.nws.products.shef.process_message_b(prod, message) → List[SHEFElement]¶: Convert the message into an object.

pyiem.nws.products.shef.process_message_e(prod, message) → List[SHEFElement]¶

Process a text string in E SHEF format.

Parameters:

prod (SHEFProduct) – the product we are working on.
message (str) – The string to parse.

Return type:

List[SHEFElement]

pyiem.nws.products.shef.process_messages(func, prod, messages) → int¶: Safe frontend to do message processing.

pyiem.nws.products.shef.process_modifiers(text, diction, basevalid)¶

Apply modifications based on what the token is telling us.

Parameters:

text (str) – Potential new information.
diction (SHEFElement) – our current elemenet definition
basevalid (datetime) – the base valid in case of relative time.

Return type:

bool for if this text was handled.

pyiem.nws.products.shef.slash_tokenize(message)¶: Carefully discover tokens.

pyiem.nws.products.shef.strip_comments(line)¶: Remove comments.

pyiem.nws.products.sigmet module¶

Parse SIGMETs

class pyiem.nws.products.sigmet.SIGMET¶

Bases: object

Data Structure.

class pyiem.nws.products.sigmet.SIGMETProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a Storm Prediction Center Mesoscale Convective Discussion

compute_time(ddhhmi)¶: Convert this string into a proper date time

get_jabbers(uri, _uri2=None)¶: Return the Jabber for this sigmet

process_SIGC()¶: Process this type of SIGMET

process_ocean()¶: Process oceananic

sql(txn)¶: Do SQL related stuff that is required

pyiem.nws.products.sigmet.compute_esol(pts, distance)¶: Figure out the box points given the two points and the distance

pyiem.nws.products.sigmet.figure_expire(ptime, hour, minute)¶: Convert something like 0255Z into a full blown time

pyiem.nws.products.sigmet.go2lonlat(lon0, lat0, direction, displacement)¶: http://stackoverflow.com/questions/7222382

pyiem.nws.products.sigmet.locs2lonslats(loc_provider, locstr, geotype, _widthstr, diameterstr)¶: Convert a locstring into a lon lat arrays

pyiem.nws.products.sigmet.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.spacewx module¶

Space Weather Center processor

class pyiem.nws.products.spacewx.SpaceWxProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Class for parsing and representing Space Wx Products

get_jabbers(uri, _uri2=None)¶: Custom Implementation of the TextProduct#get_jabbers

pyiem.nws.products.spacewx.parser(buf, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: A parser implementation

pyiem.nws.products.spcpts module¶

Storm Prediction Center PTS Product Parser.

My life was not supposed to end like this, what a brutal format.

class pyiem.nws.products.spcpts.SPCOutlook(category, threshold, multipoly)¶

Bases: object

A class holding what we store for a single outlook.

class pyiem.nws.products.spcpts.SPCOutlookCollection(issue, expire, day)¶

Bases: object

A collection of outlooks for a single ‘day’.

add_outlook(outlook)¶: We insert an outlook in an ordered manner.

difference_geometries()¶: Do the difference work to figure out actual geometries.

get_categories()¶: Return list of categories covered in this outlook.

class pyiem.nws.products.spcpts.SPCPTS(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

A class representing the polygons and metadata in SPC PTS Product.

compute_wfos(_txn=None)¶: Figure out which WFOs are impacted by this polygon.

draw_outlooks()¶: For debugging, draw the outlooks on a simple map for inspection.

find_issue_expire()¶: Determine the period this product is valid for.

find_outlooks()¶: Find the outlook sections within the text product.

get_descript_and_url()¶: Help to convert awips id into strings.

get_jabbers(uri, _uri2=None)¶: Wordsmith the Jabber/Twitter Messaging.

get_outlook(category, threshold, day)¶: Get an outlook by category and threshold.

get_outlookcollection(day)¶: Return the SPCOutlookCollection for a given day.

set_metadata()¶: Set some metadata about this product.

sql(txn)¶

Do database work.

Parameters:: txn (psycopg.cursor) – database cursor

pyiem.nws.products.spcpts.compute_times(afos, issue, expire, day)¶

Compute actual issue, expire time.

For the multi-day products, the text product contains a range of dates that need translated to an actual issue and expire time.

Returns:

issue (datetime)
expire (datetime)

pyiem.nws.products.spcpts.get_day(prod, text)¶: Figure out which day this is for.

pyiem.nws.products.spcpts.get_segments_from_text(text)¶: Return list of segments for this text.

pyiem.nws.products.spcpts.imgsrc_from_row(row: dict) → str | None¶: Compute the SPC image source for a given database row.

pyiem.nws.products.spcpts.init_days(prod)¶: Figure out which days this product should have based on the AFOS.

pyiem.nws.products.spcpts.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Parse this text.

pyiem.nws.products.spcpts.str2multipolygon(s)¶

Convert string PTS data into a polygon.

Parameters:: s (str) – the cryptic string that we attempt to make valid polygons from

pyiem.nws.products.sps module¶

Special Weather Statement

class pyiem.nws.products.sps.SPSProduct(text, utcnow=None, ugc_provider: UGCProvider | dict | None = None, nwsli_provider=None, parse_segments=True)¶

Bases: TextProduct

A Special Weather Statement

get_jabbers(uri, _uri2=None)¶: return the standard [[text, html, xtra], ] for jabber

sql(txn)¶: Do database save in the case of a polygon

pyiem.nws.products.sps.dedup_headline(headline, ugcs, counties, expire)¶

Try to not be redundant

Parameters:

headline (str) – our current headline
ugcs (list(ugc)) – list of ugcs this SPS is for
counties (str) – our current parsed string of counties
expire (str) – our current parsed expiration string

Returns:

our new counties, which has been deduped (str): our new expire, which has been deduped

Return type:

(str)

pyiem.nws.products.sps.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: The SPS Parser

pyiem.nws.products.taf module¶

TAF Parsing

class pyiem.nws.products.taf.TAFProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

Represents a TAF

get_channels()¶: Return a list of channels

get_jabbers(uri, _uri2=None)¶: Get the jabber variant of this message

sql(txn)¶: Persist to the database.

pyiem.nws.products.taf.add_forecast_info(fx, text)¶: Common things.

pyiem.nws.products.taf.ddhhmi2valid(prod, text)¶: Figure out what valid time this is.

pyiem.nws.products.taf.make_forecast(prod, text)¶: Build a TAFForecast data model.

pyiem.nws.products.taf.make_tempo(prod, text)¶: Parse a tempo group.

pyiem.nws.products.taf.parse_prod(prod)¶: Generate a data object from this product.

pyiem.nws.products.taf.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function

pyiem.nws.products.vtec module¶

A NWS TextProduct that contains VTEC information.

class pyiem.nws.products.vtec.VTECProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

A TextProduct that contains VTEC information.

do_sbw_geometry(txn, segment, vtec)¶

Storage of Storm Based Warning geometry

The IEM uses a seperate table for the Storm Based Warning geometries.

Parameters:

txn (psycopg) – Database transaction/cursor
segment (TextProduct.TextProductSegment) – Segment
vtec (pyiem.vtec.VTEC) – VTEC instance

do_sql_vtec(txn, segment, vtec)¶

Persist the non-SBW stuff to the database

Arguments: txn – A psycopg transaction segment – A TextProductSegment instance vtec – A vtec instance

get_first_non_cancel_segment()¶: Return the first segment that is a non-CAN

get_first_non_cancel_vtec()¶: Return the first non-CANcel VTEC

get_jabbers(uri, river_uri=None)¶

Return a list of triples representing how this goes to social Arguments: uri – The URL for the VTEC Browser river_uri – The URL of the River App

Returns: [[plain, html, xtra]] – A list of triples of plain text, html, xtra

get_skip_con()¶: Should this product be skipped from generating jabber messages

is_homogeneous()¶: Test to see if this product contains just one VTEC event

is_single_action()¶: Is this product just 1 VTEC action?

sql(txn)¶

Persist to the database

Parameters:: txn (psycopg.transaction) – A database transaction object that we can exec() database calls against.

exception pyiem.nws.products.vtec.VTECProductException¶

Bases: TextProductException

Something we can raise when bad things happen!

pyiem.nws.products.vtec.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: Helper function that actually converts the raw text and emits an VTECProduct instance or returns an exception

pyiem.nws.products.wwp module¶

Parsing of Storm Prediction Center WWP Product.

class pyiem.nws.products.wwp.WWPProduct(text, utcnow=None)¶

Bases: TextProduct

Class representing a WWP Product

is_test()¶: Is this a test product?

sql(txn)¶

Do the necessary database work

Parameters:: (psycopg.transaction) – a database transaction

pyiem.nws.products.wwp.parser(text, utcnow=None, _ugc_provider=None, _nwsli_provider=None)¶

Parse SPC WWP Product.

Parameters:

text (str) – the raw text to parse
utcnow (datetime) – the current datetime with timezone set!

Returns:

WWPProduct instance

pyiem.nws.products.xteus module¶

WPC’s XTEUS Nationwide High/Low.

class pyiem.nws.products.xteus.XTEUSProduct(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶

Bases: TextProduct

A Special Weather Statement

sql(cursor)¶: Do database insert logic.

pyiem.nws.products.xteus.parser(text, utcnow=None, ugc_provider=None, nwsli_provider=None)¶: The XTEUS Parser

Module contents¶

A generalized parser frontend.

pyiem.nws.products.parser(text, utcnow=None, ugc_provider: UGCProvider | dict | None = None, nwsli_provider=None)¶

Omnibus parser of NWS Text Data

This is intended to be a catch-all parser of text data. As it currently stands, it does not correctly hand products off to the correct sub-processor, but some day it will!

Parameters:

text (str) – The actual product text, this can have the <cntr>-a character to start the string.
utcnow (datetime, optional) – What is the current time, this is useful for when ingesting old data. Many times, the product does not contain enough information to assign a current valid timestamp to it. So we need to know the current timestamp to do the relative computation.
ugc_provider (UGCProvider, optional) – Provides UGC information for product parsing.
nwsli_provider (dict, optional) – Provides NWS Location Identifiers to allow lookup of geographic information for station identifiers.

Returns:

A TextProduct instance

Return type:

TextProduct

pyiem.nws.products package¶

Submodules¶

pyiem.nws.products.cf6 module¶

pyiem.nws.products.cli module¶

pyiem.nws.products.cwa module¶

pyiem.nws.products.dsm module¶

pyiem.nws.products.ero module¶

pyiem.nws.products.fd module¶

pyiem.nws.products.ffg module¶

pyiem.nws.products.gairmet module¶

pyiem.nws.products.hml module¶

pyiem.nws.products.hwo module¶

pyiem.nws.products.lsr module¶

pyiem.nws.products.mcd module¶

pyiem.nws.products.metarcollect module¶

pyiem.nws.products.mos module¶

pyiem.nws.products.nhc module¶

pyiem.nws.products.nldn module¶

pyiem.nws.products.pirep module¶

pyiem.nws.products.saw module¶

pyiem.nws.products.scp module¶

pyiem.nws.products.sel module¶

pyiem.nws.products.shef module¶

Formats¶

Implementation Notes¶

TODO List¶

pyiem.nws.products.sigmet module¶

pyiem.nws.products.spacewx module¶

pyiem.nws.products.spcpts module¶

pyiem.nws.products.sps module¶

pyiem.nws.products.taf module¶

pyiem.nws.products.vtec module¶

pyiem.nws.products.wwp module¶

pyiem.nws.products.xteus module¶

Module contents¶

Table of Contents

Previous topic

Next topic

This Page