User:Brett Wuth/Running Notes/2016-03

From PCSAR

Revision as of 00:40, 24 March 2016 (edit) Brett Wuth (Talk | contribs) (Add Members:2016-03-01_10_34_33u-scan.pdf) ← Previous diff		Current revision (06:38, 25 August 2016) (edit) (undo) Brett Wuth (Talk | contribs) (Add Members:2016-03-18 phone.com statement.pdf)
(31 intermediate revisions not shown.)
Line 3:		Line 3:
	documents:		documents:
	* {{link|Image:Members:2016-03-01_10_34_33u-scan.pdf}}		* {{link|Image:Members:2016-03-01_10_34_33u-scan.pdf}}
		+
		+
		+	== 2016-03-05 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-05_09_52_48u-scan.pdf}}
		+
		+
		+	== 2016-03-07 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-07_09_54_45u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-07_09_55_56u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-07_12_02_49u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-07_12_07_30u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-08_05_36_04u-scan.pdf}}
		+
		+
		+	== 2016-03-14 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-14 Delorme InvoiceIR08218441.pdf}}
		+
		+
		+	== 2016-03-16 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-17_05_37_25u-scan.pdf}}
		+
		+
		+	== 2016-03-17 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-17_15_43_00u-scan.pdf}}
		+
		+
		+	== 2016-03-18 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-18 phone.com statement.pdf}}
		+
		+
		+	== 2016-03-19 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-19_12_39_41u-scan.pdf}}
	== 2016-03-23 ==		== 2016-03-23 ==
	* {{subpage|2016-03-23 Administrator|Phone meeting with Administrator}}		* {{subpage|2016-03-23 Administrator|Phone meeting with Administrator}}
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-23_10_45_08u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-23_10_48_58u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-23_10_49_55u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-23_10_54_24u-scan0013.pdf}}
		+	* {{link|Image:Members:2016-03-23_11_01_05u-scan.pdf}}
		+	== 2016-03-26 ==
		+	Susanna reports website not working.
		+
		+	wiki pages show
		+	<pre>
		+	PCSAR has a problem
		+
		+	Sorry! This site is experiencing technical difficulties.
		+
		+	Try waiting a few minutes and reloading.
		+
		+	(Can't contact the database server: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) (localhost))
		+	</pre>
		+
		+	<pre>
		+	pcsar:/var/log/apache2# ps ax | grep mysql
		+	4395 pts/0 S<+ 0:00 grep mysql
		+	pcsar:/var/log/apache2# /etc/init.d/mysql stop
		+	Stopping MySQL database server: mysqld.
		+	pcsar:/var/log/apache2# /etc/init.d/mysql start
		+	Starting MySQL database server: mysqld ..
		+	Checking for corrupt, not cleanly closed and upgrade needing tables..
		+	pcsar:/var/log/apache2# ps ax | grep mysql
		+	4440 pts/0 S< 0:00 /bin/sh /usr/bin/mysqld_safe
		+	4477 pts/0 S<l 0:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sock
		+	4478 pts/0 S< 0:00 logger -p daemon.err -t mysqld_safe -i -t mysqld
		+	4566 pts/0 S<+ 0:00 grep mysql
		+	pcsar:/var/log/apache2#
		+	</pre>
		+
		+	Mysql is not running. Starting it fixes the problem.
		+
		+	About once every 20 to 60 minutes
		+	log shows:
		+	<pre>
		+	Mar 25 14:35:58 pcsar mysqld_safe[3004]: Number of processes running now: 0
		+	Mar 25 14:35:59 pcsar mysqld_safe[3006]: restarted
		+	Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: The log sequence number in ibdata files does not match
		+	Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: the log sequence number in the ib_logfiles!
		+	Mar 25 14:36:04 pcsar mysqld[3009]: 160325 14:36:04 InnoDB: Database was not shut down normally!
		+	Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: Starting crash recovery.
		+	Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: Reading tablespace information from the .ibd files...
		+	Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: Restoring possible half-written data pages from the doublewrite
		+	Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: buffer...
		+	Mar 25 14:36:13 pcsar mysqld[3009]: 160325 14:36:13 InnoDB: Started; log sequence number 02469806575
		+	Mar 25 14:36:19 pcsar mysqld[3009]: 160325 14:36:19 [Note] /usr/sbin/mysqld: ready for connections.
		+	Mar 25 14:36:19 pcsar mysqld[3009]: Version: '5.0.51a-24+lenny5' socket: '/var/run/mysql/mysqld.sock' port: 3306 (Debian)
		+	</pre>
		+	last sequence before hang:
		+	<pre>
		+	Mar 25 15:40:42 pcsar mysqld_safe[3131]: Number of processes running now: 0
		+	Mar 25 15:40:43 pcsar mysqld_safe[3133]: restarted
		+	Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: The log sequence number in ibdata files does not match
		+	Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: the log sequence number in the ib_logfiles!
		+	Mar 25 15:40:50 pcsar mysqld[3136]: 160325 15:40:50 InnoDB: Database was not shut down normally!
		+	Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: Starting crash recovery.
		+	Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: Reading tablespace information from the .ibd files...
		+	Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: Restoring possible half-written data pages from the doublewrite
		+	Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: buffer...
		+	Mar 25 15:40:56 pcsar mysqld[3136]: 160325 15:40:56 InnoDB: Started; log sequence number 0 2469847120
		+	Mar 25 15:41:03 pcsar mysqld[3136]: 160325 15:41:03 [Note] /usr/sbin/mysqld: ready for connections.
		+	Mar 25 15:41:03 pcsar mysqld[3136]: Version: '5.0.51a-24+lenny5' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)
		+	Mar 25 15:42:22 pcsar mysqld_safe[3154]: Number of processes running now: 0
		+	Mar 25 15:42:27 pcsar mysqld_safe[3156]: restarted
		+	Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: The log sequence number in ibdata files does not match
		+	Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: the log sequence number in the ib_logfiles!
		+	Mar 25 15:42:39 pcsar mysqld[3159]: 160325 15:42:38 InnoDB: Database was not shut down normally!
		+	Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: Starting crash recovery.
		+	Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: Reading tablespace information from the .ibd files...
		+	Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: Restoring possible half-written data pages from the doublewrite
		+	Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: buffer...
		+	Mar 25 15:42:44 pcsar mysqld[3159]: 160325 15:42:44 InnoDB: Started; log sequence number 0 2469847120
		+	Mar 25 15:42:47 pcsar mysqld[3159]: 160325 15:42:47 [Note] /usr/sbin/mysqld: ready for connections.
		+	Mar 25 15:42:47 pcsar mysqld[3159]: Version: '5.0.51a-24+lenny5' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)
		+	Mar 25 15:45:26 pcsar mysqld_safe[3176]: Number of processes running now: 0
		+	Mar 25 15:45:26 pcsar mysqld_safe[3178]: restarted
		+	Mar 25 15:45:51 pcsar mysqld_safe[3183]: ended
		+	</pre>
		+	Suspect same issue as {{link|Members:Server/Web server/Issue/2016-02-27 Too much memory used}}.
		+	Processes are being killed because run out of memory. Eventually that's mysqld_safe, so no further recovery.
		+	== 2016-03-27 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-27_19_14_57u-scan.pdf}}
		+	== 2016-03-28 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-28_21_24_48u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_25_52u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_27_24u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_28_22u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_29_31u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_31_24u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_33_13u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_35_41u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_38_27u-scan0015.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_40_59u-scan0030.pdf}}
		+	* {{link|Image:Members:2016-03-28_21_43_31u-scan0045.pdf}}
		+	* {{link|Image:Members:2016-03-29_03_48_38u-scan.pdf}}
		+	* {{link|Image:Members:2016-03-29_04_10_24u-scan.pdf}}
		+	== 2016-03-30 ==
		+
		+	documents:
		+	* {{link|Image:Members:2016-03-30_14_16_46u-scan.pdf}}

---

Current revision

Contents 1 2016-03-01 2 2016-03-05 3 2016-03-07 4 2016-03-14 5 2016-03-16 6 2016-03-17 7 2016-03-18 8 2016-03-19 9 2016-03-23 10 2016-03-26 11 2016-03-27 12 2016-03-28 13 2016-03-30

[edit] 2016-03-01

documents:

• 2016-02-14 incident files - call-out

[edit] 2016-03-05

documents:

• 2016-02-24 ATB Mastercard statement

[edit] 2016-03-07

documents:

• 2016-03-01 board mtg - wuth notes
• 2016-02-02 board mtg - minutes
• 2015 T1204 RCMP
• Gaminig Information sessions mailing
• 2015-05-08 Administrator Contract Services Agreement with Diann Piwek

[edit] 2016-03-14

documents:

• Image:Members:2016-03-14 Delorme InvoiceIR08218441.pdf

[edit] 2016-03-16

documents:

• 2016-03-17 STARS agreement

[edit] 2016-03-17

documents:

• 2016-03-17 rope rescue minutes

[edit] 2016-03-18

documents:

• Image:Members:2016-03-18 phone.com statement.pdf

[edit] 2016-03-19

documents:

• 2016-03-19 wilderness exercise sign-in

[edit] 2016-03-23

• Phone meeting with Administrator

documents:

• 2016-03-18 AGLC mailing re 2013 Casino Use of Proceeds Authorization
• 2016-03-18 AGLC mailing re 2013 Casino Use of Proceeds Authorization
• 2016-03-18 AGLC mailing re 2016 Casino Use of Proceeds Authorization - pages 1-12
• 2016-03-18 AGLC mailing re 2016 Casino Use of Proceeds Authorization - pages 13-15
• 2016-03-18 AGLC mailing re 2016 Casino Use of Proceeds Authorization - envelope

[edit] 2016-03-26

Susanna reports website not working.

wiki pages show

PCSAR has a problem

Sorry! This site is experiencing technical difficulties.

Try waiting a few minutes and reloading.

(Can't contact the database server: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) (localhost))

pcsar:/var/log/apache2# ps ax | grep mysql
 4395 pts/0    S<+    0:00 grep mysql
pcsar:/var/log/apache2# /etc/init.d/mysql stop
Stopping MySQL database server: mysqld.
pcsar:/var/log/apache2# /etc/init.d/mysql start
Starting MySQL database server: mysqld ..
Checking for corrupt, not cleanly closed and upgrade needing tables..
pcsar:/var/log/apache2# ps ax | grep mysql
 4440 pts/0    S<     0:00 /bin/sh /usr/bin/mysqld_safe
 4477 pts/0    S<l    0:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sock
 4478 pts/0    S<     0:00 logger -p daemon.err -t mysqld_safe -i -t mysqld
 4566 pts/0    S<+    0:00 grep mysql
pcsar:/var/log/apache2# 

Mysql is not running. Starting it fixes the problem.

About once every 20 to 60 minutes log shows:

Mar 25 14:35:58 pcsar mysqld_safe[3004]: Number of processes running now: 0
Mar 25 14:35:59 pcsar mysqld_safe[3006]: restarted
Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: The log sequence number in ibdata files does not match
Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: the log sequence number in the ib_logfiles!
Mar 25 14:36:04 pcsar mysqld[3009]: 160325 14:36:04  InnoDB: Database was not shut down normally!
Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: Starting crash recovery.
Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: Reading tablespace information from the .ibd files...
Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: Restoring possible half-written data pages from the doublewrite
Mar 25 14:36:04 pcsar mysqld[3009]: InnoDB: buffer...
Mar 25 14:36:13 pcsar mysqld[3009]: 160325 14:36:13  InnoDB: Started; log sequence number 02469806575
Mar 25 14:36:19 pcsar mysqld[3009]: 160325 14:36:19 [Note] /usr/sbin/mysqld: ready for connections.
Mar 25 14:36:19 pcsar mysqld[3009]: Version: '5.0.51a-24+lenny5'  socket: '/var/run/mysql/mysqld.sock'  port: 3306  (Debian)

last sequence before hang:

Mar 25 15:40:42 pcsar mysqld_safe[3131]: Number of processes running now: 0
Mar 25 15:40:43 pcsar mysqld_safe[3133]: restarted
Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: The log sequence number in ibdata files does not match
Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: the log sequence number in the ib_logfiles!
Mar 25 15:40:50 pcsar mysqld[3136]: 160325 15:40:50  InnoDB: Database was not shut down normally!
Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: Starting crash recovery.
Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: Reading tablespace information from the .ibd files...
Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: Restoring possible half-written data pages from the doublewrite
Mar 25 15:40:50 pcsar mysqld[3136]: InnoDB: buffer...
Mar 25 15:40:56 pcsar mysqld[3136]: 160325 15:40:56  InnoDB: Started; log sequence number 0 2469847120
Mar 25 15:41:03 pcsar mysqld[3136]: 160325 15:41:03 [Note] /usr/sbin/mysqld: ready for connections.
Mar 25 15:41:03 pcsar mysqld[3136]: Version: '5.0.51a-24+lenny5'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Debian)
Mar 25 15:42:22 pcsar mysqld_safe[3154]: Number of processes running now: 0
Mar 25 15:42:27 pcsar mysqld_safe[3156]: restarted
Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: The log sequence number in ibdata files does not match
Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: the log sequence number in the ib_logfiles!
Mar 25 15:42:39 pcsar mysqld[3159]: 160325 15:42:38  InnoDB: Database was not shut down normally!
Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: Starting crash recovery.
Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: Reading tablespace information from the .ibd files...
Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: Restoring possible half-written data pages from the doublewrite
Mar 25 15:42:39 pcsar mysqld[3159]: InnoDB: buffer...
Mar 25 15:42:44 pcsar mysqld[3159]: 160325 15:42:44  InnoDB: Started; log sequence number 0 2469847120
Mar 25 15:42:47 pcsar mysqld[3159]: 160325 15:42:47 [Note] /usr/sbin/mysqld: ready for connections.
Mar 25 15:42:47 pcsar mysqld[3159]: Version: '5.0.51a-24+lenny5'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Debian)
Mar 25 15:45:26 pcsar mysqld_safe[3176]: Number of processes running now: 0
Mar 25 15:45:26 pcsar mysqld_safe[3178]: restarted
Mar 25 15:45:51 pcsar mysqld_safe[3183]: ended

Suspect same issue as 2016-02-27 Too much memory used. Processes are being killed because run out of memory. Eventually that's mysqld_safe, so no further recovery.

[edit] 2016-03-27

documents:

• 2015-04-13 Board Members Agreement 2015-01

[edit] 2016-03-28

documents:

• 2016-03-28 casino purchases notes
• 2015-09-24 AGLC letter
• 2015-09-24 AGLC letter
• 2015-06-26 AGLC letter of authorization for Administrator
• 2015-07-15 letter to AGLC extension request
• 2015-09-03 letter to AGLC re authorization to sign
• 2015-09-03 letter to AGLC re authorization to sign
• report to AGLC re period 2013-06-01 to 2015-05-31
• report to AGLC re period 2013-06-01 to 2015-05-31
• report to AGLC re period 2013-06-01 to 2015-05-31
• report to AGLC re period 2013-06-01 to 2015-05-31
• 2015-2016 training grant signed by PCSAR
• 2009-03-29 letter to OFC re signed training grant

[edit] 2016-03-30

documents:

• 2016-03-31 casino purchases notes