Editing Personnel Database/Processing/How To

From PCSAR

Jump to: navigation, search

Warning: You are not logged in. Your IP address will be recorded in this page's edit history.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.

Current revision Your text
Line 1: Line 1:
-
==How to process the personnel database==
+
==How to process the membership database==
Processing the database requires UNIX scripts, which I ([[Brett Wuth]]) keep
Processing the database requires UNIX scripts, which I ([[Brett Wuth]]) keep
on the PCSAR virtual machine.
on the PCSAR virtual machine.
Line 10: Line 10:
running record of the last changes they've seen.
running record of the last changes they've seen.
-
=== Pending edits ===
+
=== Extract from wiki ===
-
Check my personal copy of the call-out list to see any marked-up changes that I've noted
+
{{rbw}} to do: optimize for speed (last run 7min)
-
may be necessary.
+
-
=== Extract and validate ===
+
cd ~/sar/pc/membership/wiki-extract
-
{{rbw|as used}}
+
time ./wiki-membership-database-get >db.out
-
to do
+
mv db.out ../personnel-database.xml
-
* validate upon edit rather than wait for process
+
-
** (reverified after 2015-07 damage) enable PCSARBot to receive email on watched pages
+
-
*** configure exim to receive e-mail locally
+
-
*** login as PCSARBot
+
-
*** change e-mail address to bot@pcsar.castrov.cuug.ab.ca
+
-
*** confirm new address
+
-
*** enable e-mail options for
+
-
**** E-mail me when a page I'm watching is changed
+
-
**** E-mail me also for minor edits of pages
+
-
** (reverified after 2015-07 damage) validate Activities upon edit
+
-
*** Add [[Activities]] to PCSARBot's watchlist
+
-
*** uses .procmailrc
+
-
*** uses script ~bot/sar/pc/membership/bin/wiki-membership-activities-validate to update [[Activities/Report/wiki-membership-activities-validate]]
+
-
*** amend script to output errors with contextual lines
+
-
** validate each Personnel Record upon edit
+
-
*** (reverified after 2015-07 damage) write script to validate just member passed
+
-
**** see ~bot/sar/pc/membership/bin/wiki-membership-member-validate
+
-
**** Use "{{member|Brett Wuth}}" as experiment
+
-
*** (reverified after 2015-07 damage) update all memrec pages to include validation report
+
-
**** define as part of memrec template
+
-
*** develop trigger on memrec changes
+
-
**** develop dispatch on changes to individual memrec or Activities
+
-
***** (recheck) create trial watch on my mem record ({{member|Brett Wuth}})
+
-
****** (done) Add PCSARBot to "TrustedGuest" group
+
-
***** ''currently uses exact match on page name in .procmailrc
+
-
***** (done) verify changes to Brett Wuth's MemRec triggers report update
+
-
***** develop mechanism to register dispatch for page, what script to run
+
-
****** ''on wiki itself
+
-
****** ''advantage: inspectability: easy for others self, to see what is going on
+
-
****** ''disadvantage: more work
+
-
****** ''single page [[Members:PCSARBot/Watches]]
+
-
******* ''each line like:'' <code><nowiki>* [[pagename]]: script</nowiki></code>
+
-
******* ''with possible duplicates of pagename, specifying different scripts
+
-
****** (done) write scripts to enable/disable watch on a wiki page
+
-
******* (done) wiki-page-watch-enable
+
-
******* (done) wiki-page-watch-disable
+
-
****** (done) write scripts to add/remove watch scripts
+
-
******* (done) wiki-watch-script-add
+
-
******* (done) wiki-watch-script-remove
+
-
****** '''(next)''' create mechanism to dispatch scripts based on email of page that changed; <code>wiki-watch-scripts-dispatch</code>
+
-
******* ''consider security of other person editing page
+
-
******* (done) create tool to determine revision number of page: ~/bin/wiki-page-revision-number-get
+
-
******* (done) create tool to determine editor of particular revision of page (not necessarily current revision)
+
-
******** (done) create tool <code>wiki-page-editors-list</code> that generates list of revision id's and editors
+
-
******** (done) create script wiki-page-editor that produces editor of latest or specified revision
+
-
******* (done) create tool to get the synced content and last editor of a page: <code>wiki-page-get-with-editor</code>
+
-
******** (done) create <code>wiki-page-get</code>
+
-
******** ''consider usage pattern get-latest-revision-number; get-page; verify lasest-revision-number unchanged
+
-
******* validate against whitelist of last editors
+
-
******** '''design choice:''' use single hardcoded name: 'Brett Wuth'
+
-
******** consider: can whitelist be stored on wiki page? That page would need to be validated itself. Could be rooted in a single hardcoded name.
+
-
******** shortcut: validate against single hardcoded name until need list
+
-
****** create mechanism to unwatch pages if email came in and no watch script registered
+
-
**** develop watch on memrec category
+
-
***** add new pages
+
-
***** delete removed pages
+
-
*** distribute previously created tools through git
+
-
*** allow Members: page to transclude report from another Members: page
+
-
*** when [[Activities]] changes, reprocess all Members
+
-
**** only if no errors in Activities itself -- otherwise error would appear on each page
+
-
**** only file report if change -- otherwise many unrelated updates
+
-
{{rbw|as used}} to do: optimize for speed (last run 7min)
+
=== Merging Edits to Personnel Records ===
-
 
+
-
as user "wuth"
+
cd ~/sar/pc/membership
cd ~/sar/pc/membership
-
wiki-extract
 
 +
check in the changes:
 +
rcsdiff personnel-database.xml
 +
ci -l personnel-database.xml
 +
 +
=== validate ===
 +
 +
* Validate Edits to Personnel Records
 +
validate
 +
(uses less) or
 +
validation-report
 +
Check my marked up copy of the call out list to see any changes
 +
that may be necessary.
* Reconcile
* Reconcile
make any necessary changes.
make any necessary changes.
-
=== Process ===
+
=== check in ===
-
xslt <personnel-database.xml lib/Applicants.xsl | xslt lib/Name.xsl
+
wuth@pcsar:~/sar/pc/membership$ ci -l personnel-database.xml
 +
=== Tagged Percent ===
-
* <big>'''make sure current ssh-agent authority is extended'''</big>
+
* Prepare the TaggedPercent version:
-
ssh -AX wuth@localhost
+
-
database-process
+
RecordTaggedPercentFieldsDefault -v FieldsFile=lib/RecordTaggedPercentFields.txt <tmp/personnel-database.not-validated >tmp/personnel-database.txt
 +
=== Clean up old print jobs ===
 +
rm run/print-queue/*
-
* prints copies for myself
+
=== call-out list ===
-
* sends personnel list (names only) to [[Members:Protocols/Shell H2S/Global Training|Global Training for validation of H2S Alive course registration.]]
+
* To Do:
 +
** commonize elements of {bin,lib}/{call-out,first-call}
-
Manually check the Unicode in reports:
+
* To print Call-out list for RCMP, [[FireHall]], Recorder, and CallOut Coordinator
-
* [[Members:Membership Database/PCSAR]]
+
call-out-list-print
-
* [[Members:Membership Database/Combined Members]]
+
-
* [[Members:Call Out/List]]
+
-
==== D4H export ====
+
-
2021-03-21 D4H has changed SSL protocols. Requires upgraded version python.
+
-
d4h-web-interface.py user:pcsard4hbot@pcsar.ca.d4h.org membersImport
+
=== first-call list ===
 +
* Print and deliver copy to Fire Hall, RCMP
 +
first-call-list-print
-
may need to re-download database to get newly assigned D4HMemberIDs
 
-
wiki-extract
 
-
d4h-web-interface.py user:pcsard4hbot@pcsar.ca.d4h.org activitiesAttendanceImport
+
* Fax to Fire Dispatch in Lethbridge -- not done any more
 +
first-call-list-fax
-
==== Rave export ====
+
=== personnel records ===
-
wuth@pcsar:~/sar/pc/membership$ call-out-list-xml >tmp/call-out-list.xml
+
* To print the individual personnel sheets:
-
wuth@desk:~/wuth/pcremo/rave/rave-api-tools$ scp -p wuth@pcsar.castrov.cuug.ab.ca:sar/pc/membership/tmp/call-out-list.xml .
+
xslt <personnel-database.xml lib/MembersAndApplicants.xsl | changes-print recorder
-
wuth@desk:~/wuth/pcremo/rave/rave-api-tools$ python3 -m sar
+
xslt <personnel-database.xml lib/MembersOnly.xsl | changes-print SARCAR
 +
** not RCMP because they no longer keep records
-
** sometimes fails at initial log in and has to be restarted
+
=== mailing lists ===
-
** takes ~65 minutes to run
+
* To update the pcsar-board/pcsar-preplan mailing lists:
-
*** see {{subpage|Rapid Notify}}
+
mailing-lists-update
-
==== Continue ====
+
=== e-mailed reports ===
-
* To Do: {{rbw|as used}}
+
 
 +
* e-mail to [[Training Coordinator]]:
 +
training-coordinator-send
 +
 
 +
* Print/Send the Fan-Out list -- no longer done
 +
fan-out-list-send
 +
 
 +
* Report on automatically detected policy issues:
** additional policy checks
** additional policy checks
-
*** write policy check for sar fund currency
 
-
**** Set type of existing Activities. See /[ssh/pcsar]/home/wuth/sar/pc/membership/lib/Membership.xsd ActivityCategoryType (done to 2012-01)
 
-
**** base on /[scp/pcsar]/home/wuth/sar/pc/membership/bin/policy-check-stale
 
-
***** see lib/SARFundStale.xsl
 
-
****** needs debugging
 
-
**** add to CVS
 
*** write policy check for voting member upgrade
*** write policy check for voting member upgrade
*** write policy check for voting member currency
*** write policy check for voting member currency
-
** commonize elements of {bin,lib}/{call-out,first-call}
+
*** write policy check for sar fund currency
 +
policy-checks-email
 +
* To E-Mail to SARA:
 +
sara-send
-
* Personnel records: -- done by Adam now
+
* To E-Mail to the agencies:
-
lpr -Pofficejet8500 -o outputorder=reverse run/print-queue/*{recorder,SARCAR}*
+
first-call-send
-
* Fax to Fire Dispatch in Lethbridge -- not done any more
+
* To E-Mail the call-out list:
-
first-call-list-fax
+
call-out-list-send
 +
=== printing ===
 +
'''E-mail job to Adam for printing.'''
-
* Print/Send the Fan-Out list -- no longer done
 
-
fan-out-list-send
 
-
* Call Out List: -- done by Adam
+
* Personnel records:
 +
lpr -Pofficejet8500 -o outputorder=reverse run/print-queue/*{recorder,SARCAR}*
 +
* First Call List:
 +
** 1 copy (for binder)
 +
lpr -Pofficejet8500 run/print-queue/*FirstCall*
 +
* Call Out List:
** for binder
** for binder
** for SARCAR
** for SARCAR
** for overhead box at firehall
** for overhead box at firehall
-
** 3 copies
+
** for myself
 +
** 4 copies
 +
lpr -#4 -Pofficejet8500 -o Collate=True -o outputorder=reverse run/print-queue/*CallOutList*
 +
* To print the print queue:
 +
print-queue-email-bob
 +
** (to Bob Costa)
 +
** or
 +
print-queue-email
 +
** (to Roy Davidson)
-
=== sync wiki accounts ===
+
=== etc ===
 +
* produce a list of members that haven't been active in last 6 months for membership committee
-
* [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list sync] the personnel list with the wiki user list
 
-
** rename/merge Charlene/Charleen Bruder
 
-
** Extract list of users currently with "Member" status
 
-
*** http://pcsar.dyndns.org:8080/mediawiki/index.php?title=Special:Listusers&limit=500&group=Members
 
-
wuth@pcsar:~/sar/pc/membership$
+
* E-mail board list of member applicants
-
xslt lib/MembersOnly.xsl < personnel-database.xml | xslt lib/EMailPrimaryTxtWithNone.xsl >tmp/address-list
+
-
cat /home/wuth/sar/pc/membership/tmp/address-list | sed -e 's|\(.*\) <\(.*\)>|sudo /usr/local/bin/wiki-members-add "\1" "\2"|' | bash
+
-
will require manual manipulation to
+
<pre>
 +
To: Pincher Creek SAR Board <pcsar-board@castrov.cuug.ab.ca>
 +
Subject: membership
 +
--text follows this line--
 +
Hi All,
-
'''To Do:
+
This month there are two returning people that want to rejoin PCSAR:
-
* '''(next)''' create tool to drop member of group
+
Wanda Burchert and Claus Burchert.
-
* rename existing user
+
-
see [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list personal notes]
+
http://pcsar.dyndns.org:8080/mediawiki/index.php/Members:Claus_Burchert
 +
http://pcsar.dyndns.org:8080/mediawiki/index.php/Members:Wanda_Burchert
 +
 
 +
Please let me know if their memberships are approved. When you make a
 +
decision, could you please inform me by e-mail; just letting me know
 +
in person during the training session, doesn't work very well because
 +
I don't always remember.
 +
 
 +
Take Care,
 +
--
 +
Brett Wuth wuth@castrov.cuug.ab.ca wuth@acm.org
 +
Box 1251-U, Pincher Creek, Alberta T0K 1W0, CANADA Tel:+1 403 627-2460
 +
OpenPGP FingerPrint=628F C9DA BDBC 2A0E 18F1 2F6A 3300 8422 BE6A 0E79
 +
What is the meaning of life?! Yes.
 +
</pre>
 +
 
 +
 
 +
* [[:Image:Members:Call Out List.pdf|Upload the Call Out List]] to the wiki.
 +
 
 +
<pre>
 +
wuth@fiore:~/tmp$
 +
scp -p pcsar:sar/pc/membership/run/print-queue/*CallOutList* .
 +
wiki-upload-doc pcsar 2011-03*CallOutList* Members:Call_Out_List.pdf ""
 +
</pre>
 +
 
 +
 
 +
* [[:Image:Members:First Call List.pdf|Upload the First Call List]] to the wiki.
 +
<pre>
 +
wuth@fiore:~/tmp$ scp -p pcsar:sar/pc/membership/run/print-queue/*FirstCall* .
 +
wuth@fiore:~/tmp$ wiki-upload-doc pcsar 2011-03*FirstCall* Members:First_Call_List.pdf ""
 +
</pre>
 +
 
 +
* [http://iweb.castrov.cuug.ab.ca/mediawiki/index.php/Mediawiki#Syncing_user_list sync] the membership list with the wiki user list
 +
** rename/merge Charlene/Charleen Bruder
==Non-monthly processing==
==Non-monthly processing==
Line 226: Line 222:
--text follows this line--
--text follows this line--
Thanks for joining Pincher Creek SAR. Can you please check if I've correctly
Thanks for joining Pincher Creek SAR. Can you please check if I've correctly
-
copied your personnel record:
+
copied your membership record:
---cut---
---cut---

Please note that all contributions to PCSAR are considered to be released under the Attribution-Share Alike 3.0 Unported (see PCSAR:Copyrights for details). If you don't want your writing to be edited mercilessly and redistributed at will, then don't submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. DO NOT SUBMIT COPYRIGHTED WORK WITHOUT PERMISSION!


Cancel | Editing help (opens in new window)

Templates used on this page:

Retrieved from "http://pcsar.dyndns.org:8080/mediawiki/index.php/Personnel_Database/Processing/How_To"
Personal tools