Current revision |
Your text |
Line 1: |
Line 1: |
- | ==How to process the personnel database== | + | ==How to process the membership database== |
| Processing the database requires UNIX scripts, which I ([[Brett Wuth]]) keep | | Processing the database requires UNIX scripts, which I ([[Brett Wuth]]) keep |
- | on the PCSAR virtual machine. | + | on my laptop. |
| + | I'm migrating them to the PCSAR virtual machine. |
| | | |
| Unless you understand UNIX, you're not likely to understand much | | Unless you understand UNIX, you're not likely to understand much |
Line 9: |
Line 10: |
| The scripts check for changes in the database, so they keep a | | The scripts check for changes in the database, so they keep a |
| running record of the last changes they've seen. | | running record of the last changes they've seen. |
| + | This running |
| + | record is currently on [[Brett Wuth]]'s laptop. |
| | | |
- | === Pending edits ===
| + | The Membership Coordinator e-mails me the database as |
- | Check my personal copy of the call-out list to see any marked-up changes that I've noted
| + | personnel-database.xml. [[Brett Wuth]] is currently doing the data |
- | may be necessary.
| + | entry on behalf of the vacant Membership Coordinator position. |
| | | |
- | === Extract and validate ===
| |
- | {{rbw|as used}}
| |
- | to do
| |
- | * validate upon edit rather than wait for process
| |
- | ** (reverified after 2015-07 damage) enable PCSARBot to receive email on watched pages
| |
- | *** configure exim to receive e-mail locally
| |
- | *** login as PCSARBot
| |
- | *** change e-mail address to bot@pcsar.castrov.cuug.ab.ca
| |
- | *** confirm new address
| |
- | *** enable e-mail options for
| |
- | **** E-mail me when a page I'm watching is changed
| |
- | **** E-mail me also for minor edits of pages
| |
- | ** (reverified after 2015-07 damage) validate Activities upon edit
| |
- | *** Add [[Activities]] to PCSARBot's watchlist
| |
- | *** uses .procmailrc
| |
- | *** uses script ~bot/sar/pc/membership/bin/wiki-membership-activities-validate to update [[Activities/Report/wiki-membership-activities-validate]]
| |
- | *** amend script to output errors with contextual lines
| |
- | ** validate each Personnel Record upon edit
| |
- | *** (reverified after 2015-07 damage) write script to validate just member passed
| |
- | **** see ~bot/sar/pc/membership/bin/wiki-membership-member-validate
| |
- | **** Use "{{member|Brett Wuth}}" as experiment
| |
- | *** (reverified after 2015-07 damage) update all memrec pages to include validation report
| |
- | **** define as part of memrec template
| |
- | *** develop trigger on memrec changes
| |
- | **** develop dispatch on changes to individual memrec or Activities
| |
- | ***** (recheck) create trial watch on my mem record ({{member|Brett Wuth}})
| |
- | ****** (done) Add PCSARBot to "TrustedGuest" group
| |
- | ***** ''currently uses exact match on page name in .procmailrc
| |
- | ***** (done) verify changes to Brett Wuth's MemRec triggers report update
| |
- | ***** develop mechanism to register dispatch for page, what script to run
| |
- | ****** ''on wiki itself
| |
- | ****** ''advantage: inspectability: easy for others self, to see what is going on
| |
- | ****** ''disadvantage: more work
| |
- | ****** ''single page [[Members:PCSARBot/Watches]]
| |
- | ******* ''each line like:'' <code><nowiki>* [[pagename]]: script</nowiki></code>
| |
- | ******* ''with possible duplicates of pagename, specifying different scripts
| |
- | ****** (done) write scripts to enable/disable watch on a wiki page
| |
- | ******* (done) wiki-page-watch-enable
| |
- | ******* (done) wiki-page-watch-disable
| |
- | ****** (done) write scripts to add/remove watch scripts
| |
- | ******* (done) wiki-watch-script-add
| |
- | ******* (done) wiki-watch-script-remove
| |
- | ****** '''(next)''' create mechanism to dispatch scripts based on email of page that changed; <code>wiki-watch-scripts-dispatch</code>
| |
- | ******* ''consider security of other person editing page
| |
- | ******* (done) create tool to determine revision number of page: ~/bin/wiki-page-revision-number-get
| |
- | ******* (done) create tool to determine editor of particular revision of page (not necessarily current revision)
| |
- | ******** (done) create tool <code>wiki-page-editors-list</code> that generates list of revision id's and editors
| |
- | ******** (done) create script wiki-page-editor that produces editor of latest or specified revision
| |
- | ******* (done) create tool to get the synced content and last editor of a page: <code>wiki-page-get-with-editor</code>
| |
- | ******** (done) create <code>wiki-page-get</code>
| |
- | ******** ''consider usage pattern get-latest-revision-number; get-page; verify lasest-revision-number unchanged
| |
- | ******* validate against whitelist of last editors
| |
- | ******** '''design choice:''' use single hardcoded name: 'Brett Wuth'
| |
- | ******** consider: can whitelist be stored on wiki page? That page would need to be validated itself. Could be rooted in a single hardcoded name.
| |
- | ******** shortcut: validate against single hardcoded name until need list
| |
- | ****** create mechanism to unwatch pages if email came in and no watch script registered
| |
- | **** develop watch on memrec category
| |
- | ***** add new pages
| |
- | ***** delete removed pages
| |
- | *** distribute previously created tools through git
| |
- | *** allow Members: page to transclude report from another Members: page
| |
- | *** when [[Activities]] changes, reprocess all Members
| |
- | **** only if no errors in Activities itself -- otherwise error would appear on each page
| |
- | **** only file report if change -- otherwise many unrelated updates
| |
| | | |
- | {{rbw|as used}} to do: optimize for speed (last run 7min)
| + | Copy my working directory from my laptop to the PCSAR |
| + | '''only if''' my laptop is more up to date. |
| + | wuth@macbook:~/sar/pc/membership$ rsync -avz . wuth@pcsar:sar/pc/membership |
| | | |
- | as user "wuth"
| + | |
| + | === Merging Edits to Personnel Records === |
| | | |
| cd ~/sar/pc/membership | | cd ~/sar/pc/membership |
- | wiki-extract
| |
| | | |
| + | I may have made changes to my copy of the database at the same |
| + | time as Bob has. The changes need to be merged. |
| | | |
| + | use: |
| + | rlog personnel-database.xml | head -20 |
| + | to find out what version I gave to Bob most recently (usually |
| + | latest). |
| | | |
- | * Reconcile
| + | check in my changes: |
- | make any necessary changes.
| + | rcsdiff personnel-database.xml |
| + | ci -l personnel-database.xml |
| + | for 1.18 use whatever version Bob based his changes on: |
| + | c ~/tmp/personnel-database.xml . |
| + | rcsdiff -r1.18 personnel-database.xml |
| + | to see his changes |
| | | |
- | === Process ===
| + | make my edits to his changes. |
- | xslt <personnel-database.xml lib/Applicants.xsl | xslt lib/Name.xsl
| + | |
| | | |
| + | rcsmerge -r1.18 personnel-database.xml |
| | | |
- | * <big>'''make sure current ssh-agent authority is extended'''</big> | + | * resolve any conflicts: |
- | ssh -AX wuth@localhost
| + | |
| | | |
- | database-process | + | rcsdiff personnel-database.xml |
| + | to verify only his changes have been propogated |
| | | |
- | * prints copies for myself
| + | === validate === |
- | * sends personnel list (names only) to [[Members:Protocols/Shell H2S/Global Training|Global Training for validation of H2S Alive course registration.]]
| + | |
| | | |
- | Manually check the Unicode in reports:
| + | * Validate Edits to Personnel Records |
- | * [[Members:Membership Database/PCSAR]] | + | validate |
- | * [[Members:Membership Database/Combined Members]]
| + | (uses less) or |
- | * [[Members:Call Out/List]]
| + | validation-report |
- | ==== D4H export ====
| + | |
- | 2021-03-21 D4H has changed SSL protocols. Requires upgraded version python.
| + | |
| | | |
- | d4h-web-interface.py user:pcsard4hbot@pcsar.ca.d4h.org membersImport
| + | Check my marked up copy of the call out list to see any changes |
| + | that may be necessary. |
| | | |
- | may need to re-download database to get newly assigned D4HMemberIDs
| + | * Reconcile |
- | wiki-extract
| + | make any necessary changes. |
| | | |
- | d4h-web-interface.py user:pcsard4hbot@pcsar.ca.d4h.org activitiesAttendanceImport
| + | === check in === |
| | | |
- | ==== Rave export ====
| + | Update the "includes changes" date comment at start of the |
- | wuth@pcsar:~/sar/pc/membership$ call-out-list-xml >tmp/call-out-list.xml
| + | file: |
- | wuth@desk:~/wuth/pcremo/rave/rave-api-tools$ scp -p wuth@pcsar.castrov.cuug.ab.ca:sar/pc/membership/tmp/call-out-list.xml .
| + | |
- | wuth@desk:~/wuth/pcremo/rave/rave-api-tools$ python3 -m sar
| + | |
| | | |
- | ** sometimes fails at initial log in and has to be restarted
| + | wuth@pcsar:~/sar/pc/membership$ ci -l personnel-database.xml |
- | ** takes ~65 minutes to run
| + | |
- | *** see {{subpage|Rapid Notify}}
| + | |
| | | |
- | ==== Continue ==== | + | === Tagged Percent === |
- | * To Do: {{rbw|as used}} | + | |
- | ** additional policy checks
| + | * Prepare the TaggedPercent version: |
- | *** write policy check for sar fund currency
| + | |
- | **** Set type of existing Activities. See /[ssh/pcsar]/home/wuth/sar/pc/membership/lib/Membership.xsd ActivityCategoryType (done to 2012-01)
| + | RecordTaggedPercentFieldsDefault -v FieldsFile=lib/RecordTaggedPercentFields.txt <tmp/personnel-database.not-validated >tmp/personnel-database.txt |
- | **** base on /[scp/pcsar]/home/wuth/sar/pc/membership/bin/policy-check-stale
| + | === Clean up old print jobs === |
- | ***** see lib/SARFundStale.xsl
| + | rm run/print-queue/* |
- | ****** needs debugging
| + | |
- | **** add to CVS | + | === call-out list === |
- | *** write policy check for voting member upgrade
| + | * To Do: |
- | *** write policy check for voting member currency | + | |
| ** commonize elements of {bin,lib}/{call-out,first-call} | | ** commonize elements of {bin,lib}/{call-out,first-call} |
| | | |
| + | * To print Call-out list for RCMP, [[FireHall]], Recorder, and CallOut Coordinator |
| + | wuth@pcsar:~/sar/pc/membership$ call-out-list-print |
| + | |
| + | === first-call list === |
| + | * Print and deliver copy to Fire Hall, RCMP |
| + | wuth@pcsar:~/sar/pc/membership$ first-call-list-print |
| | | |
- | * Personnel records: -- done by Adam now
| |
- | lpr -Pofficejet8500 -o outputorder=reverse run/print-queue/*{recorder,SARCAR}*
| |
| | | |
| * Fax to Fire Dispatch in Lethbridge -- not done any more | | * Fax to Fire Dispatch in Lethbridge -- not done any more |
| first-call-list-fax | | first-call-list-fax |
| | | |
| + | === personnel records === |
| + | * To print the individual personnel sheets: |
| + | wuth@pcsar:~/sar/pc/membership$ xslt <personnel-database.xml lib/MembersAndApplicants.xsl | changes-print recorder |
| + | wuth@pcsar:~/sar/pc/membership$ xslt <personnel-database.xml lib/MembersOnly.xsl | changes-print SARCAR |
| + | ** not RCMP because they no longer keep records |
| + | |
| + | === mailing lists === |
| + | * To update the pcsar-board/pcsar-preplan mailing lists: |
| + | wuth@pcsar:~/sar/pc/membership$ mailing-lists-update |
| + | |
| + | === e-mailed reports === |
| + | |
| + | * e-mail to [[Training Coordinator]]: |
| + | wuth@pcsar:~/sar/pc/membership$ training-coordinator-send |
| | | |
| * Print/Send the Fan-Out list -- no longer done | | * Print/Send the Fan-Out list -- no longer done |
| fan-out-list-send | | fan-out-list-send |
| | | |
- | * Call Out List: -- done by Adam | + | * Report on automatically detected policy issues: |
- | ** for binder
| + | wuth@pcsar:~/sar/pc/membership$ policy-checks-email |
- | ** for SARCAR
| + | |
- | ** for overhead box at firehall
| + | |
- | ** 3 copies
| + | |
| | | |
- | === sync wiki accounts ===
| + | * To E-Mail to SARA: |
| + | wuth@pcsar:~/sar/pc/membership$ sara-send |
| | | |
- | * [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list sync] the personnel list with the wiki user list | + | * To E-Mail to the agencies: |
- | ** rename/merge Charlene/Charleen Bruder
| + | wuth@pcsar:~/sar/pc/membership$ first-call-send |
- | ** Extract list of users currently with "Member" status
| + | |
- | *** http://pcsar.dyndns.org:8080/mediawiki/index.php?title=Special:Listusers&limit=500&group=Members
| + | |
| | | |
- | wuth@pcsar:~/sar/pc/membership$ | + | * To E-Mail the call-out list: |
- | xslt lib/MembersOnly.xsl < personnel-database.xml | xslt lib/EMailPrimaryTxtWithNone.xsl >tmp/address-list
| + | wuth@pcsar:~/sar/pc/membership$ call-out-list-send |
- | cat /home/wuth/sar/pc/membership/tmp/address-list | sed -e 's|\(.*\) <\(.*\)>|sudo /usr/local/bin/wiki-members-add "\1" "\2"|' | bash
| + | |
| | | |
- | will require manual manipulation to
| + | === printing === |
| + | * Personnel records: |
| + | lpr -Pofficejet-pcsar -o outputorder=reverse run/print-queue/*{recorder,SARCAR}* |
| + | * First Call List: |
| + | ** 1 copy (for binder) |
| + | lpr -Pofficejet-pcsar run/print-queue/*FirstCall* |
| + | * Call Out List: |
| + | ** for binder |
| + | ** for SARCAR |
| + | ** for overhead box at firehall |
| + | ** for myself |
| + | ** 4 copies |
| + | lpr -#4 -Pofficejet-pcsar -o Collate=True -o outputorder=reverse run/print-queue/*CallOutList* |
| + | * To print the print queue: |
| + | print-queue-email-bob |
| + | ** (to Bob Costa) |
| + | ** or |
| + | print-queue-email |
| + | ** (to Roy Davidson) |
| | | |
- | '''To Do:
| |
- | * '''(next)''' create tool to drop member of group
| |
- | * rename existing user
| |
| | | |
- | see [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list personal notes]
| + | |
| + | === etc === |
| + | * E-mail board list of member applicants |
| | | |
| ==Non-monthly processing== | | ==Non-monthly processing== |
Line 179: |
Line 154: |
| them haven't been done in long time and their instructions are now | | them haven't been done in long time and their instructions are now |
| out of date. | | out of date. |
- |
| |
- | === copy to Brett's laptop ===
| |
- |
| |
- | Because the remaining steps don't run on the PCSAR virtual machine yet, the file must be copied to Brett's laptop and run from there.
| |
- | <pre>
| |
- | wuth@macbook:~/sar/pc/membership$ chmod 600 personnel-database.xml
| |
- | wuth@macbook:~/sar/pc/membership$ scp -p wuth@pcsar:sar/pc/membership/personnel-database.xml .
| |
- | wuth@macbook:~/sar/pc/membership$ chmod 400 personnel-database.xml
| |
- | wuth@macbook:~/sar/pc/membership$ scp -p wuth@pcsar:sar/pc/membership/tmp/personnel-database.txt tmp
| |
- | wuth@macbook:~/sar/pc/membership$ rm -r run
| |
- | wuth@macbook:~/sar/pc/membership$ scp -pr wuth@pcsar:sar/pc/membership/run .
| |
- | </pre>
| |
| | | |
| ===To prepare a list of addresses and phone numbers of all our members=== | | ===To prepare a list of addresses and phone numbers of all our members=== |
Line 226: |
Line 189: |
| --text follows this line-- | | --text follows this line-- |
| Thanks for joining Pincher Creek SAR. Can you please check if I've correctly | | Thanks for joining Pincher Creek SAR. Can you please check if I've correctly |
- | copied your personnel record: | + | copied your membership record: |
| | | |
| ---cut--- | | ---cut--- |
Line 241: |
Line 204: |
| Alberta T0K 1W0, CANADA Tel:+1 403 627-2460 pgp-public-keys@uit.no | | Alberta T0K 1W0, CANADA Tel:+1 403 627-2460 pgp-public-keys@uit.no |
| has mine; Print=E4F8EDEECBE01AD2FA3D8B2D94B1A292 What is the meaning of life?! Yes. | | has mine; Print=E4F8EDEECBE01AD2FA3D8B2D94B1A292 What is the meaning of life?! Yes. |
- | </pre> | + | <pre> |
| ===To print the participation dates=== | | ===To print the participation dates=== |
| xslt <personnel-database.xml Activities.xsl | awk -v FS=":" {printf "%-24s:%-7s:%s\n", $1, $2, $3 } | mail -s "activity history" `MailRCAliasAddressGet "Bob Costa"` wuth | | xslt <personnel-database.xml Activities.xsl | awk -v FS=":" {printf "%-24s:%-7s:%s\n", $1, $2, $3 } | mail -s "activity history" `MailRCAliasAddressGet "Bob Costa"` wuth |