Current revision |
Your text |
Line 1: |
Line 1: |
- | ==How to process the personnel database== | + | ==How to process the membership database== |
| Processing the database requires UNIX scripts, which I ([[Brett Wuth]]) keep | | Processing the database requires UNIX scripts, which I ([[Brett Wuth]]) keep |
| on the PCSAR virtual machine. | | on the PCSAR virtual machine. |
Line 11: |
Line 11: |
| | | |
| === Pending edits === | | === Pending edits === |
- | Check my personal copy of the call-out list to see any marked-up changes that I've noted | + | Check my marked up copy of the call out list to see any changes |
- | may be necessary. | + | that may be necessary. |
| | | |
- | === Extract and validate ===
| |
- | {{rbw|as used}}
| |
- | to do
| |
- | * validate upon edit rather than wait for process
| |
- | ** (reverified after 2015-07 damage) enable PCSARBot to receive email on watched pages
| |
- | *** configure exim to receive e-mail locally
| |
- | *** login as PCSARBot
| |
- | *** change e-mail address to bot@pcsar.castrov.cuug.ab.ca
| |
- | *** confirm new address
| |
- | *** enable e-mail options for
| |
- | **** E-mail me when a page I'm watching is changed
| |
- | **** E-mail me also for minor edits of pages
| |
- | ** (reverified after 2015-07 damage) validate Activities upon edit
| |
- | *** Add [[Activities]] to PCSARBot's watchlist
| |
- | *** uses .procmailrc
| |
- | *** uses script ~bot/sar/pc/membership/bin/wiki-membership-activities-validate to update [[Activities/Report/wiki-membership-activities-validate]]
| |
- | *** amend script to output errors with contextual lines
| |
- | ** validate each Personnel Record upon edit
| |
- | *** (reverified after 2015-07 damage) write script to validate just member passed
| |
- | **** see ~bot/sar/pc/membership/bin/wiki-membership-member-validate
| |
- | **** Use "{{member|Brett Wuth}}" as experiment
| |
- | *** (reverified after 2015-07 damage) update all memrec pages to include validation report
| |
- | **** define as part of memrec template
| |
- | *** develop trigger on memrec changes
| |
- | **** develop dispatch on changes to individual memrec or Activities
| |
- | ***** (recheck) create trial watch on my mem record ({{member|Brett Wuth}})
| |
- | ****** (done) Add PCSARBot to "TrustedGuest" group
| |
- | ***** ''currently uses exact match on page name in .procmailrc
| |
- | ***** (done) verify changes to Brett Wuth's MemRec triggers report update
| |
- | ***** develop mechanism to register dispatch for page, what script to run
| |
- | ****** ''on wiki itself
| |
- | ****** ''advantage: inspectability: easy for others self, to see what is going on
| |
- | ****** ''disadvantage: more work
| |
- | ****** ''single page [[Members:PCSARBot/Watches]]
| |
- | ******* ''each line like:'' <code><nowiki>* [[pagename]]: script</nowiki></code>
| |
- | ******* ''with possible duplicates of pagename, specifying different scripts
| |
- | ****** (done) write scripts to enable/disable watch on a wiki page
| |
- | ******* (done) wiki-page-watch-enable
| |
- | ******* (done) wiki-page-watch-disable
| |
- | ****** (done) write scripts to add/remove watch scripts
| |
- | ******* (done) wiki-watch-script-add
| |
- | ******* (done) wiki-watch-script-remove
| |
- | ****** '''(next)''' create mechanism to dispatch scripts based on email of page that changed; <code>wiki-watch-scripts-dispatch</code>
| |
- | ******* ''consider security of other person editing page
| |
- | ******* (done) create tool to determine revision number of page: ~/bin/wiki-page-revision-number-get
| |
- | ******* (done) create tool to determine editor of particular revision of page (not necessarily current revision)
| |
- | ******** (done) create tool <code>wiki-page-editors-list</code> that generates list of revision id's and editors
| |
- | ******** (done) create script wiki-page-editor that produces editor of latest or specified revision
| |
- | ******* (done) create tool to get the synced content and last editor of a page: <code>wiki-page-get-with-editor</code>
| |
- | ******** (done) create <code>wiki-page-get</code>
| |
- | ******** ''consider usage pattern get-latest-revision-number; get-page; verify lasest-revision-number unchanged
| |
- | ******* validate against whitelist of last editors
| |
- | ******** '''design choice:''' use single hardcoded name: 'Brett Wuth'
| |
- | ******** consider: can whitelist be stored on wiki page? That page would need to be validated itself. Could be rooted in a single hardcoded name.
| |
- | ******** shortcut: validate against single hardcoded name until need list
| |
- | ****** create mechanism to unwatch pages if email came in and no watch script registered
| |
- | **** develop watch on memrec category
| |
- | ***** add new pages
| |
- | ***** delete removed pages
| |
- | *** distribute previously created tools through git
| |
- | *** allow Members: page to transclude report from another Members: page
| |
- | *** when [[Activities]] changes, reprocess all Members
| |
- | **** only if no errors in Activities itself -- otherwise error would appear on each page
| |
- | **** only file report if change -- otherwise many unrelated updates
| |
| | | |
- | {{rbw|as used}} to do: optimize for speed (last run 7min) | + | === Extract from wiki === |
| + | {{rbw}} to do: optimize for speed (last run 7min) |
| | | |
- | as user "wuth"
| + | cd ~/sar/pc/membership/wiki-extract |
| + | time ./wiki-membership-database-get >db.out |
| + | mv db.out ../personnel-database.xml |
| + | |
| + | === Merging Edits to Personnel Records === |
| | | |
| cd ~/sar/pc/membership | | cd ~/sar/pc/membership |
- | wiki-extract
| |
| | | |
| + | check in the changes: |
| + | rcsdiff personnel-database.xml |
| + | ci -l personnel-database.xml |
| + | |
| + | === validate === |
| | | |
| + | * Validate Edits to Personnel Records |
| + | validate |
| + | (uses less) or |
| + | validation-report |
| | | |
| * Reconcile | | * Reconcile |
| make any necessary changes. | | make any necessary changes. |
| | | |
- | === Process === | + | === check in === |
- | xslt <personnel-database.xml lib/Applicants.xsl | xslt lib/Name.xsl | + | wuth@pcsar:~/sar/pc/membership$ ci -l personnel-database.xml |
| | | |
| + | === Tagged Percent === |
| | | |
- | * <big>'''make sure current ssh-agent authority is extended'''</big> | + | * Prepare the TaggedPercent version: |
- | ssh -AX wuth@localhost
| + | |
| | | |
- | database-process
| + | RecordTaggedPercentFieldsDefault -v FieldsFile=lib/RecordTaggedPercentFields.txt <tmp/personnel-database.not-validated >tmp/personnel-database.txt |
| + | === Clean up old print jobs === |
| + | rm run/print-queue/* |
| | | |
- | * prints copies for myself
| + | === call-out list === |
- | * sends personnel list (names only) to [[Members:Protocols/Shell H2S/Global Training|Global Training for validation of H2S Alive course registration.]] | + | * To Do: |
| + | ** commonize elements of {bin,lib}/{call-out,first-call} |
| | | |
- | Manually check the Unicode in reports:
| + | * To print Call-out list for RCMP, [[FireHall]], Recorder, and CallOut Coordinator |
- | * [[Members:Membership Database/PCSAR]] | + | call-out-list-print |
- | * [[Members:Membership Database/Combined Members]]
| + | |
- | * [[Members:Call Out/List]]
| + | |
- | ==== D4H export ====
| + | |
- | 2021-03-21 D4H has changed SSL protocols. Requires upgraded version python.
| + | |
| | | |
- | d4h-web-interface.py user:pcsard4hbot@pcsar.ca.d4h.org membersImport | + | === first-call list === |
| + | * Print and deliver copy to Fire Hall, RCMP |
| + | first-call-list-print |
| | | |
- | may need to re-download database to get newly assigned D4HMemberIDs
| |
- | wiki-extract
| |
| | | |
- | d4h-web-interface.py user:pcsard4hbot@pcsar.ca.d4h.org activitiesAttendanceImport | + | * Fax to Fire Dispatch in Lethbridge -- not done any more |
| + | first-call-list-fax |
| | | |
- | ==== Rave export ==== | + | === personnel records === |
- | wuth@pcsar:~/sar/pc/membership$ call-out-list-xml >tmp/call-out-list.xml
| + | * To print the individual personnel sheets: |
- | wuth@desk:~/wuth/pcremo/rave/rave-api-tools$ scp -p wuth@pcsar.castrov.cuug.ab.ca:sar/pc/membership/tmp/call-out-list.xml .
| + | xslt <personnel-database.xml lib/MembersAndApplicants.xsl | changes-print recorder |
- | wuth@desk:~/wuth/pcremo/rave/rave-api-tools$ python3 -m sar
| + | xslt <personnel-database.xml lib/MembersOnly.xsl | changes-print SARCAR |
| + | ** not RCMP because they no longer keep records |
| | | |
- | ** sometimes fails at initial log in and has to be restarted
| + | === mailing lists === |
- | ** takes ~65 minutes to run | + | * To update the pcsar-board/pcsar-preplan mailing lists: |
- | *** see {{subpage|Rapid Notify}}
| + | mailing-lists-update |
| | | |
- | ==== Continue ==== | + | === e-mailed reports === |
- | * To Do: {{rbw|as used}} | + | |
| + | * e-mail to [[Training Coordinator]]: |
| + | training-coordinator-send |
| + | |
| + | * Print/Send the Fan-Out list -- no longer done |
| + | fan-out-list-send |
| + | |
| + | * Report on automatically detected policy issues: |
| ** additional policy checks | | ** additional policy checks |
- | *** write policy check for sar fund currency
| |
- | **** Set type of existing Activities. See /[ssh/pcsar]/home/wuth/sar/pc/membership/lib/Membership.xsd ActivityCategoryType (done to 2012-01)
| |
- | **** base on /[scp/pcsar]/home/wuth/sar/pc/membership/bin/policy-check-stale
| |
- | ***** see lib/SARFundStale.xsl
| |
- | ****** needs debugging
| |
- | **** add to CVS
| |
| *** write policy check for voting member upgrade | | *** write policy check for voting member upgrade |
| *** write policy check for voting member currency | | *** write policy check for voting member currency |
- | ** commonize elements of {bin,lib}/{call-out,first-call} | + | *** write policy check for sar fund currency |
| + | policy-checks-email |
| | | |
| + | * To E-Mail to SARA: |
| + | sara-send |
| | | |
- | * Personnel records: -- done by Adam now | + | * To E-Mail to the agencies: |
- | lpr -Pofficejet8500 -o outputorder=reverse run/print-queue/*{recorder,SARCAR}* | + | first-call-send |
| | | |
- | * Fax to Fire Dispatch in Lethbridge -- not done any more | + | * To E-Mail the call-out list: |
- | first-call-list-fax | + | call-out-list-send |
| | | |
| + | === printing === |
| + | '''E-mail job to Adam for printing.''' |
| | | |
- | * Print/Send the Fan-Out list -- no longer done
| |
- | fan-out-list-send
| |
| | | |
- | * Call Out List: -- done by Adam | + | * Personnel records: |
| + | lpr -Pofficejet8500 -o outputorder=reverse run/print-queue/*{recorder,SARCAR}* |
| + | * First Call List: |
| + | ** 1 copy (for binder) |
| + | lpr -Pofficejet8500 run/print-queue/*FirstCall* |
| + | * Call Out List: |
| ** for binder | | ** for binder |
| ** for SARCAR | | ** for SARCAR |
| ** for overhead box at firehall | | ** for overhead box at firehall |
- | ** 3 copies | + | ** for myself |
| + | ** 4 copies |
| + | lpr -#4 -Pofficejet8500 -o Collate=True -o outputorder=reverse run/print-queue/*CallOutList* |
| + | * To print the print queue: |
| + | print-queue-email |
| + | ** (to Adam) |
| + | ** or |
| + | print-queue-email-bob |
| + | ** (to Bob Costa) |
| | | |
- | === sync wiki accounts === | + | === etc === |
| | | |
- | * [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list sync] the personnel list with the wiki user list | + | * [[:Image:Members:Call Out List.pdf|Upload the Call Out List]] to the wiki. |
- | ** rename/merge Charlene/Charleen Bruder
| + | |
- | ** Extract list of users currently with "Member" status
| + | |
- | *** http://pcsar.dyndns.org:8080/mediawiki/index.php?title=Special:Listusers&limit=500&group=Members
| + | |
| | | |
- | wuth@pcsar:~/sar/pc/membership$
| + | <pre> |
- | xslt lib/MembersOnly.xsl < personnel-database.xml | xslt lib/EMailPrimaryTxtWithNone.xsl >tmp/address-list
| + | wuth@fiore:~/tmp$ |
- | cat /home/wuth/sar/pc/membership/tmp/address-list | sed -e 's|\(.*\) <\(.*\)>|sudo /usr/local/bin/wiki-members-add "\1" "\2"|' | bash
| + | scp -p pcsar:sar/pc/membership/run/print-queue/*CallOutList* . |
| + | wiki-upload-doc pcsar 2011-03*CallOutList* Members:Call_Out_List.pdf "" |
| + | </pre> |
| | | |
- | will require manual manipulation to
| |
| | | |
- | '''To Do:
| + | * [[:Image:Members:First Call List.pdf|Upload the First Call List]] to the wiki. |
- | * '''(next)''' create tool to drop member of group | + | <pre> |
- | * rename existing user | + | wuth@fiore:~/tmp$ |
| + | scp -p pcsar:sar/pc/membership/run/print-queue/*FirstCall* . |
| + | wiki-upload-doc pcsar 2011-03*FirstCall* Members:First_Call_List.pdf "" |
| + | </pre> |
| | | |
- | see [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list personal notes]
| + | * [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list sync] the membership list with the wiki user list |
| + | ** rename/merge Charlene/Charleen Bruder |
| | | |
| ==Non-monthly processing== | | ==Non-monthly processing== |
Line 226: |
Line 195: |
| --text follows this line-- | | --text follows this line-- |
| Thanks for joining Pincher Creek SAR. Can you please check if I've correctly | | Thanks for joining Pincher Creek SAR. Can you please check if I've correctly |
- | copied your personnel record: | + | copied your membership record: |
| | | |
| ---cut--- | | ---cut--- |