mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   No Prime Left Behind (https://www.mersenneforum.org/forumdisplay.php?f=82)
-   -   Server outrages (https://www.mersenneforum.org/showthread.php?t=13840)

gd_barnes 2017-03-21 21:54

There was a massive fire yesterday in a new apt. complex that was under construction less than a half mile from me while I was out of town. They shut off the electricity in the surrounding neighborhoods. Several houses in the neighborhood north of me burned and the wind was blowing strongly out of the north towards the south and my neighborhood. The houses that burned were only about a quarter mile to my north. Fortunately I was already scheduled to come back today and my house was not affected. Electricity was restored late yesterday and I got home an hour ago and just now got the servers running again.

Google "Overland Park KS fire" if you want to read about it. Below is one link. It was an 8-alarm fire and was the biggest in Overland Park history. (Overland Park is a suburb of Kansas City, pop. 100,000-150,000.) Even fire fighters from Kansas City over 10 miles away responded. I don't think they've determined what started it yet but several people said they heard a big boom so maybe an explosion. I feel very fortunate.

[URL]http://www.kmbc.com/article/fire-rips-through-overland-park-apartment-complex/9158681[/URL]

MyDogBuster 2017-03-21 21:59

[QUOTE] I feel very fortunate. [/QUOTE]

Glad everything was okay. Fires are scary, unpredictable.

MisterBitcoin 2017-03-22 17:01

[QUOTE=gd_barnes;455262]There was a massive fire yesterday in a new apt. complex that was under construction less than a half mile from me while I was out of town. They shut off the electricity in the surrounding neighborhoods. Several houses in the neighborhood north of me burned and the wind was blowing strongly out of the north towards the south and my neighborhood. The houses that burned were only about a quarter mile to my north. Fortunately I was already scheduled to come back today and my house was not affected. Electricity was restored late yesterday and I got home an hour ago and just now got the servers running again.

Google "Overland Park KS fire" if you want to read about it. Below is one link. It was an 8-alarm fire and was the biggest in Overland Park history. (Overland Park is a suburb of Kansas City, pop. 100,000-150,000.) Even fire fighters from Kansas City over 10 miles away responded. I don't think they've determined what started it yet but several people said they heard a big boom so maybe an explosion. I feel very fortunate.

[URL]http://www.kmbc.com/article/fire-rips-through-overland-park-apartment-complex/9158681[/URL][/QUOTE]

I saw an video about it in the german media. That was a massive fire, I hope that no one got hurt.

AMDave 2017-05-23 13:30

A couple of months ago the NPLB DR (disaster recovery) server started "throwing its toys out of the cradle".

During the DR server outages backups were still being captured at the DR location but not restored successfully to the DR server.

The NPLB DR server now sits inside a comfortable "Redback" 12U cabinet with master cooling and a new PSU has has been installed.
(personal opinion - I think the server room is now several degrees cooler than it used to be - old PSUs should be herded out to the back paddock and dealt with appropriately :edit: )

No further problems have occurred over the last 2 months. The DR recovery process has been re-tested several times successfully and the NPLB DR server is now updated to the last restore as at "Last update: 2017-05-22 02:07:14" including all databases, log files and candidates.

rebirther 2017-05-23 18:14

[QUOTE=AMDave;459596]A couple of months ago the NPLB DR (disaster recovery) server started "throwing its toys out of the cradle".

During the DR server outages backups were still being captured at the DR location but not restored successfully to the DR server.

The NPLB DR server now sits inside a comfortable "Redback" 12U cabinet with master cooling and a new PSU has has been installed.
(personal opinion - I think the server room is now several degrees cooler than it used to be - old PSUs should be herded out to the back paddock and dealt with appropriately :edit: )

No further problems have occurred over the last 2 months. The DR recovery process has been re-tested several times successfully and the NPLB DR server is now updated to the last restore as at "Last update: 2017-05-22 02:07:14" including all databases, log files and candidates.[/QUOTE]

Cant reach the server anymore.

AMDave 2017-05-23 21:47

Me neither.
The last known IP address is unreponsive, but I can tell you that the last log message I got from the server was timestamped at "23/05/2017 11:05:50 "

gd_barnes 2017-05-24 04:06

I had a power blip. It is back up now.

rebirther 2017-05-24 21:33

down again :/

gd_barnes 2017-05-24 22:10

Back up. Connectivity issue.

henryzz 2017-05-25 09:37

It seems that the DR server is in a more reliable location than the main server.

mdettweiler 2017-06-30 13:28

The noprimeleftbehind.net server appears to have suffered a hard reboot overnight. (The website was up and running fine, but the PRPnet servers were all down. The servers' lock files had not been cleaned up indicating the shutdown was hard.)

I've restarted all the PRPnet servers as of a few minutes ago.

AMDave 2017-06-30 23:31

Thanks Max!

I just finished migrating my own servers to IPv6 DDNS

The old address for the NPLB DR machine is no more (nplb.no-ip.org is now defunct)

The NPLB DR host is now at [URL="http://noprimeleftbehind.dynv6.net"]noprimeleftbehind.dynv6.net[/URL]

gd_barnes 2017-07-01 21:38

I just now got home from out of town. Thanks Max for taking care of that! :smile:

juhehe 2017-11-26 10:14

Some things are down?
 
[url]www.noprimeleftbehind.net[/url] is not working for me (Finland/Europe).

Recent results in [url]http://noprimeleftbehind.dynv6.net/stats/index.php?content=recent[/url]
are in the state of:
Last update: 2017-11-17 02:11:22
Server Time: 2017-11-26 19:14:18

My result was the last one in...

gd_barnes 2017-11-26 10:53

I may have a hard drive issue. The machine shut down and I cannot get it to reboot. I will look more into it on Sunday afternoon.

All data is backed up relatively frequently so any loss of data would be minimal.

gd_barnes 2017-11-26 23:58

I cannot get the machine to boot. Dave, what is the status of the backup?

gd_barnes 2017-11-27 06:09

Well, I determined that the problem is not the hard drive. In the various attempts at rebooting the machine I had messed up the boot area. I was able to run the fsck -v command and the computer booted up fine.

I then started the servers and it ran for a while. It accepted all work units that it was waiting on, sent out new ones, and then proceeded to reboot itself again for no particular reason.

I'm now thinking I have a bad motherboard or possibly a bad power supply. I will try replacing the power supply. I have a few spares laying around.

gd_barnes 2017-11-27 10:23

After replacing the power supply the machine still wants to intermittently reboot itself. Sometimes it goes a few hours, sometimes only a few minutes.

I'm guessing now that it is the motherboard.

So...I'll keep the servers running as best as I can until I can get the motherboard replaced.

gd_barnes 2017-11-29 03:37

I have moved the server machine hard drive to another machine and it no longer reboots itself. I confirmed that the new machine has internet access. I started the servers but nothing happens. No work units are being received or sent. Also CRUS's and NPLB's web pages are not accessible from the normal links on our projects.

The machine is very similar to the previous server machine. Slightly older motherboard and CPU both very similar.

I feel like there is something wrong in the internet setup even though the machine connects fine to the internet. Does anyone have any ideas?

AMDave 2017-11-29 19:38

Apologies for the delay in responding. I did get sent the notification of your post but it was languishing in my spam filter.

The server is still sending me the log files so outbound comms is working and therefore I also have the IP address. We just need to fix the inbound routing, so that may be either on the router or DynDNS.

I will be back home tonight so I'll get a good look at it then.

Last restore on the DR machine was "Last update: 2017-11-17 02:11:22"
[url]http://noprimeleftbehind.dynv6.net/stats/[/url]
although I am still having an issue with the DynDNS automatic update
but otherwise all good there.

AMDave 2017-11-30 12:14

It appears that the NPLB server is updating the dynamic DNS route correctly.
I have confirmed that the server is reporting the correct address and is updating the dynamic DNS index with the correct address.
I have to suggest that there is a setting on the router connected to the server that is the issue.
Perhaps the port-forward rule is bound to the old MAC address of NIC on the old motherboard.
I cannot resolve it from here.
If Max has some time he may be able to help check the router config,

gd_barnes 2017-12-01 04:00

I sent a PM to Max asking for his help.

I looked at the router config on the new machine and everything looks fine to me. Obviously something needs to be changed somewhere.

AMDave 2017-12-01 12:28

when the connection is restored I need to do some DB maintenance to restore the stats.
The main table is busted due to HDD write corruption when the mobo hicupped

Here is the excerpt from the logfile that I get.

[CODE]Completed local_port_status at Fri Dec 1 06:00:09 2017

DBD::mysql::st execute failed: Table './nplbstats/results' is marked as crashed and last (automatic?) repair failed at /home/nplb/scripts/refresh_database.pl line 476.
DBD::mysql::st execute failed: Table './nplbstats/results' is marked as crashed and last (automatic?) repair failed at /home/nplb/scripts/refresh_database.pl line 675.
DBD::mysql::st execute failed: Table './nplbstats/results' is marked as crashed and last (automatic?) repair failed at /home/nplb/scripts/refresh_database.pl line 805.
Starting refresh_database at Fri Dec 1 06:00:09 2017[/CODE]

mdettweiler 2017-12-02 01:56

Dave's guess was exactly right, it's a silly router issue: the port-forwards for the Internet-facing servers are bound to the MAC address of the Ethernet port on Jeepford's old motherboard. (We've been using a DHCP reservation instead of directly configuring Jeepford's static IP ever since we set up Gary's new router a year or two ago.)

I can fix this easily, but to do so I need access to the router's configuration page, which would be easy if not for the fact that (due to this issue) I don't have remote access to Gary's network. :smile: So, I've PM'd him directions to download and run a handy-dandy little utility called "[url=https://ngrok.com[/url]ngrok[/url]" which will bypass the router and create a direct tunnel to Jeepford, which I can use to get in and fix this.

gd_barnes 2017-12-02 07:27

OK I just got home. I will begin doing what Max requested. I will update this post if everything goes successfully.

Once Max has access to my "new" Jeepford machine I have no doubt that he will be able to fix the problem quickly.

Edit: Max, everything that you asked for in the PM has been done. Let me know if I need to do anything else.

mdettweiler 2017-12-03 04:58

All fixed! :smile: The router configuration has been updated and everything is once again reachable from the Internet. I see that clients have already happily connected to the PRPnet servers and gotten some long-awaited tests.

Gary, I'll send you some directions (with screenshots!) for how to do this yourself in case it ever happens again. I'll also send them to Dave. The fewer maintenance tasks that "only one of us can do", the better. :smile:

gd_barnes 2017-12-03 09:00

[QUOTE=mdettweiler;473016]All fixed! :smile: The router configuration has been updated and everything is once again reachable from the Internet. I see that clients have already happily connected to the PRPnet servers and gotten some long-awaited tests.

Gary, I'll send you some directions (with screenshots!) for how to do this yourself in case it ever happens again. I'll also send them to Dave. The fewer maintenance tasks that "only one of us can do", the better. :smile:[/QUOTE]

Oustanding! Thank you Max!

odicin 2017-12-06 07:41

@Dave: It seems that the NPLB port report for all 3 ports stuck at 2017-09-29.

Regards Odi

AMDave 2017-12-07 09:13

Hi Odi
/me waves
That's correct.
I have confirmed control of the stats server is restored, but it is a non-trivial task to restore the broken results table from the last backup before it broke, reprocess the result files up to the point where it broke & then process the data that has accumulated.
I work in a different city to where I live so I opted to defer it to this weekend when I can get a good run at it with my mind on the tasks.
However, your contributions are being recorded correctly and will be included in the stats when I restore the processes in the next day or two.

AMDave 2017-12-12 12:03

Apologies fro the stats blip.
[url]http://stats.free-dc.org/stats.php?page=proj&proj=nplb[/url]
The hardware issue has been resolved and the corrupted database table has been rectified.
Move along. Nothing to see here.
[url]https://www.youtube.com/watch?v=pdFl__NlOpA[/url]

AMDave 2017-12-12 13:18

NPLB stats database refresh rescheduled from every hour to 3-hourly to improve resource utilisation.

rebirther 2017-12-12 20:06

Some CRUS tables are also not updating and are empty.

gd_barnes 2017-12-12 21:51

[QUOTE=rebirther;473876]Some CRUS tables are also not updating and are empty.[/QUOTE]

Dave,

I noticed this too. The first time I noticed it was a couple of days ago. I don't know if it is related to your change but the timing does seem coincidental. Can you check into this? There are CRUS tables that have previously been updating every hour at 30 minutes after the hour. Here are the bad tables:

[URL]http://www.noprimeleftbehind.net/crus/vstats_new/crus-stats.htm[/URL]
[URL]http://www.noprimeleftbehind.net/crus/vstats_new/crus-top20.htm[/URL]
[URL]http://www.noprimeleftbehind.net/crus/vstats_new/crus-unproven.htm[/URL]
[URL]http://www.noprimeleftbehind.net/crus/vstats_new/crus-proven.htm[/URL]

Strangely this table is OK:
[url]http://www.noprimeleftbehind.net/crus/tab/CRUS_tab.htm[/url]

The links to the above five pages are on this page:
[URL]http://www.noprimeleftbehind.net/crus/[/URL]

AMDave 2017-12-13 21:40

Thanks for the report. I will have a look and see what I can do.

gd_barnes 2017-12-14 07:45

[QUOTE=AMDave;473954]Thanks for the report. I will have a look and see what I can do.[/QUOTE]

I found the problem on my end. For some reason the machine had lost its internet connection although the servers continued working just fine. I stopped the servers, rebooted it, restarted the servers, and the internet connection worked just fine. Subsequently at the next run time at 30 minutes after the hour, the stats pages reran and now look all good.

AMDave 2017-12-14 09:40

Thanks.
I am still chasing down some NPLB db performance issues.

odicin 2017-12-15 18:35

Today I noticed, the overall stats [URL]http://www.noprimeleftbehind.net/stats/index.php?content=participant_stats[/URL] starts in 10/17 and so I jumped in third position ;)

Regards Odi

AMDave 2017-12-15 23:08

Thanks again Odi. I hadn't spotted that yet.
I have more to do tonight when I get home.

AMDave 2017-12-17 10:14

The NPLB results summary has been repaired.
Sorry Odi.:reality-check:

AMDave 2017-12-20 03:01

NPLB stats are running optimally again.
The regular hourly stats update schedule has been reinstated.

The Carnivore 2019-01-12 22:38

The server appears to be down:
[url]https://downforeveryoneorjustme.com/www.noprimeleftbehind.net[/url]

gd_barnes 2019-01-13 04:28

It was down for 2-3 hours due to a power blip and a stubborn ethernet splitter. It's been back up for several hours now.

masser 2019-04-14 14:36

Down last night and this morning:

[URL="https://downforeveryoneorjustme.com/noprimeleftbehind.net"]https://downforeveryoneorjustme.com/noprimeleftbehind.net[/URL]

gd_barnes 2019-04-15 02:33

Server and ports are back up. Sorry about the outage.

MooMoo2 2019-05-05 22:58

There is something strange going on.

[url]http://noprimeleftbehind.net/tps/[/url] is up and running, but

[url]http://noprimeleftbehind.net:12000/all.html[/url] is down, and I can't get any work from port 12000.

gd_barnes 2019-05-06 00:47

The server machine continues to reboot itself. It may be a day before I can get it fixed.

gd_barnes 2019-05-06 20:25

Bad power supply in server machine. Replaced. Everything looks good now. :smile:

gd_barnes 2020-01-09 23:53

The NPLB and CRUS PRPnet servers are down and have been for over 12 hours.

It is not a computer issue. The NPLB and CRUS web pages are OK and I am able to update them.

I am getting a MySQL error that I am unable to fix when I try to start a PRPnet server. It is:
Connect to database failed: [MySQL][ODBC 3.51 Driver]Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2), native code=2002

It is an old Linux machine.

I have tried stopping and restarting MySQL with:
sudo service mysql stop
sudo service mysql start

The stop works fine. When it tries to start it fails with:
* Starting MySQL database server mysqld [ OK ]
* Checking for corrupt, not cleanly closed and upgrade needing tables.
gary@jeepford:~$ ERROR 2013 (HY000) at line 2: Lost connection to MySQL server during query

I have shut down the machine and restarted several times. A few times the PRPnet servers would run for awhile and then shut down. Other times they would not run at all.

I would appreciate any help.

unconnected 2020-01-10 06:09

Gary, check mysqld log for errors, usually it is located at /var/log/mysqld.log. Also it make sense to check *.err files in the mysql data directory (/var/lib/mysql usually). Check that you have enough disk space and free ram.

If there are errors about db/tables corruption in logs, run the next command to check: 'mysqlcheck --all-databases --check --verbose'. This command only checks db structure/integrity, for fix use with --repair option (it is good idea to backup data directory first).
Hope this helps.

gd_barnes 2020-01-10 08:09

Dmitry,

Here is a screen shot of my recent attempt to do what you suggested:

[code]
gary@jeepford:~$ sudo service mysql status
[sudo] password for gary:
* MySQL is stopped.
gary@jeepford:~$ sudo service mysql start
* Starting MySQL database server mysqld [ OK ]
* Checking for corrupt, not cleanly closed and upgrade needing tables.
gary@jeepford:~$ ERROR 2013 (HY000) at line 2: Lost connection to MySQL server during query

gary@jeepford:~$ mysqlcheck --all-databases
mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) when trying to connect
gary@jeepford:~$ mysqlcheck --all-databases --repair
mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) when trying to connect
gary@jeepford:~$
[/code]

As you can see I cannot execute the commands because I cannot connect. Any suggestions?

I'm ignorant about MySQL.

unconnected 2020-01-10 08:18

Ah, mysqlcheck command is useless if mysqld service is not even run. Gary, what you see in the log files?

gd_barnes 2020-01-10 08:39

Here are the last few pages. I hope this is enough.

[code]
Jan 10 02:34:22 jeepford mysqld[8351]: This could be because you hit a bug. It is also possible that this binary
Jan 10 02:34:22 jeepford mysqld[8351]: or one of the libraries it was linked against is corrupt, improperly built,
Jan 10 02:34:22 jeepford mysqld[8351]: or misconfigured. This error can also be caused by malfunctioning hardware.
Jan 10 02:34:22 jeepford mysqld[8351]: We will try our best to scrape up some info that will hopefully help diagnose
Jan 10 02:34:22 jeepford mysqld[8351]: the problem, but since we have already crashed, something is definitely wrong
Jan 10 02:34:22 jeepford mysqld[8351]: and this may fail.
Jan 10 02:34:22 jeepford mysqld[8351]:
Jan 10 02:34:22 jeepford mysqld[8351]: key_buffer_size=134217728
Jan 10 02:34:22 jeepford mysqld[8351]: read_buffer_size=6291456
Jan 10 02:34:22 jeepford mysqld[8351]: max_used_connections=0
Jan 10 02:34:22 jeepford mysqld[8351]: max_connections=200
Jan 10 02:34:22 jeepford mysqld[8351]: threads_connected=0
Jan 10 02:34:22 jeepford mysqld[8351]: It is possible that mysqld could use up to
Jan 10 02:34:22 jeepford mysqld[8351]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2588672 K
Jan 10 02:34:22 jeepford mysqld[8351]: bytes of memory
Jan 10 02:34:22 jeepford mysqld[8351]: Hope that's ok; if not, decrease some variables in the equation.
Jan 10 02:34:22 jeepford mysqld[8351]:
Jan 10 02:34:22 jeepford mysqld[8351]: thd=(nil)
Jan 10 02:34:22 jeepford mysqld[8351]: Attempting backtrace. You can use the following information to find out
Jan 10 02:34:22 jeepford mysqld[8351]: where mysqld died. If you see no messages after this, something went
Jan 10 02:34:22 jeepford mysqld[8351]: terribly wrong...
Jan 10 02:34:22 jeepford mysqld[8351]: frame pointer is NULL, did you compile with
Jan 10 02:34:22 jeepford mysqld[8351]: -fomit-frame-pointer? Aborting backtrace!
Jan 10 02:34:22 jeepford mysqld[8351]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Jan 10 02:34:22 jeepford mysqld[8351]: information that should help you find out what is causing the crash.
Jan 10 02:34:22 jeepford mysqld_safe[8368]: Number of processes running now: 0
Jan 10 02:34:22 jeepford mysqld_safe[8370]: restarted
Jan 10 02:34:22 jeepford mysqld[8373]: InnoDB: Log scan progressed past the checkpoint lsn 65 3180302167
Jan 10 02:34:22 jeepford mysqld[8373]: 200110 2:34:22 InnoDB: Database was not shut down normally!
Jan 10 02:34:22 jeepford mysqld[8373]: InnoDB: Starting crash recovery.
Jan 10 02:34:22 jeepford mysqld[8373]: InnoDB: Reading tablespace information from the .ibd files...
Jan 10 02:34:22 jeepford mysqld[8373]: InnoDB: Restoring possible half-written data pages from the doublewrite
Jan 10 02:34:22 jeepford mysqld[8373]: InnoDB: buffer...
Jan 10 02:34:22 jeepford mysqld[8373]: InnoDB: Doing recovery: scanned up to log sequence number 65 3185544704
Jan 10 02:34:22 jeepford mysqld[8373]: InnoDB: Doing recovery: scanned up to log sequence number 65 3186966404
Jan 10 02:34:22 jeepford mysqld[8373]: 200110 2:34:22 InnoDB: Starting an apply batch of log records to the database...
Jan 10 02:34:46 jeepford mysqld[8373]: InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Jan 10 02:34:46 jeepford mysqld[8373]: InnoDB: Apply batch completed
Jan 10 02:34:46 jeepford mysqld[8373]: 200110 2:34:46 InnoDB: Started; log sequence number 65 3186966404
Jan 10 02:34:46 jeepford mysqld[8373]: 200110 2:34:46 [Note] /usr/sbin/mysqld: ready for connections.
Jan 10 02:34:46 jeepford mysqld[8373]: Version: '5.0.75-0ubuntu10.5' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: Error: trying to access page number 1768430057 in space 0,
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: space name ./ibdata1,
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: which is outside the tablespace bounds.
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: Byte offset 0, len 16384, i/o type 10.
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: If you get this error at mysqld startup, please check that
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: your my.cnf matches the ibdata files that you have in the
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: MySQL server.
Jan 10 02:34:47 jeepford mysqld[8373]: 200110 2:34:47InnoDB: Assertion failure in thread 140683719387472 in file fil0fil.c line 3959
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: We intentionally generate a memory trap.
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: If you get repeated assertion failures or crashes, even
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: immediately after the mysqld startup, there may be
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: corruption in the InnoDB tablespace. Please refer to
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
Jan 10 02:34:47 jeepford mysqld[8373]: InnoDB: about forcing recovery.
Jan 10 02:34:47 jeepford mysqld[8373]: 200110 2:34:47 - mysqld got signal 11 ;
Jan 10 02:34:47 jeepford mysqld[8373]: This could be because you hit a bug. It is also possible that this binary
Jan 10 02:34:47 jeepford mysqld[8373]: or one of the libraries it was linked against is corrupt, improperly built,
Jan 10 02:34:47 jeepford mysqld[8373]: or misconfigured. This error can also be caused by malfunctioning hardware.
Jan 10 02:34:47 jeepford mysqld[8373]: We will try our best to scrape up some info that will hopefully help diagnose
Jan 10 02:34:47 jeepford mysqld[8373]: the problem, but since we have already crashed, something is definitely wrong
Jan 10 02:34:47 jeepford mysqld[8373]: and this may fail.
Jan 10 02:34:47 jeepford mysqld[8373]:
Jan 10 02:34:47 jeepford mysqld[8373]: key_buffer_size=134217728
Jan 10 02:34:47 jeepford mysqld[8373]: read_buffer_size=6291456
Jan 10 02:34:47 jeepford mysqld[8373]: max_used_connections=0
Jan 10 02:34:47 jeepford mysqld[8373]: max_connections=200
Jan 10 02:34:47 jeepford mysqld[8373]: threads_connected=0
Jan 10 02:34:47 jeepford mysqld[8373]: It is possible that mysqld could use up to
Jan 10 02:34:47 jeepford mysqld[8373]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2588672 K
Jan 10 02:34:47 jeepford mysqld[8373]: bytes of memory
Jan 10 02:34:47 jeepford mysqld[8373]: Hope that's ok; if not, decrease some variables in the equation.
Jan 10 02:34:47 jeepford mysqld[8373]:
Jan 10 02:34:47 jeepford mysqld[8373]: thd=(nil)
Jan 10 02:34:47 jeepford mysqld[8373]: Attempting backtrace. You can use the following information to find out
Jan 10 02:34:47 jeepford mysqld[8373]: where mysqld died. If you see no messages after this, something went
Jan 10 02:34:47 jeepford mysqld[8373]: terribly wrong...
Jan 10 02:34:47 jeepford mysqld[8373]: frame pointer is NULL, did you compile with
Jan 10 02:34:47 jeepford mysqld[8373]: -fomit-frame-pointer? Aborting backtrace!
Jan 10 02:34:47 jeepford mysqld[8373]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Jan 10 02:34:47 jeepford mysqld[8373]: information that should help you find out what is causing the crash.
Jan 10 02:34:48 jeepford mysqld_safe[8395]: Number of processes running now: 0
Jan 10 02:34:48 jeepford mysqld_safe[8397]: restarted
Jan 10 02:34:48 jeepford mysqld[8400]: InnoDB: Log scan progressed past the checkpoint lsn 65 3180302167
Jan 10 02:34:48 jeepford mysqld[8400]: 200110 2:34:48 InnoDB: Database was not shut down normally!
Jan 10 02:34:48 jeepford mysqld[8400]: InnoDB: Starting crash recovery.
Jan 10 02:34:48 jeepford mysqld[8400]: InnoDB: Reading tablespace information from the .ibd files...
Jan 10 02:34:48 jeepford mysqld[8400]: InnoDB: Restoring possible half-written data pages from the doublewrite
Jan 10 02:34:48 jeepford mysqld[8400]: InnoDB: buffer...
Jan 10 02:34:48 jeepford mysqld[8400]: InnoDB: Doing recovery: scanned up to log sequence number 65 3185544704
Jan 10 02:34:48 jeepford mysqld[8400]: InnoDB: Doing recovery: scanned up to log sequence number 65 3186966404
Jan 10 02:34:48 jeepford mysqld[8400]: 200110 2:34:48 InnoDB: Starting an apply batch of log records to the database...
Jan 10 02:35:13 jeepford mysqld[8400]: InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Jan 10 02:35:13 jeepford mysqld[8400]: InnoDB: Apply batch completed
Jan 10 02:35:13 jeepford mysqld[8400]: 200110 2:35:13 InnoDB: Started; log sequence number 65 3186966404
Jan 10 02:35:13 jeepford mysqld[8400]: 200110 2:35:13 [Note] /usr/sbin/mysqld: ready for connections.
Jan 10 02:35:13 jeepford mysqld[8400]: Version: '5.0.75-0ubuntu10.5' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: Error: trying to access page number 1768430057 in space 0,
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: space name ./ibdata1,
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: which is outside the tablespace bounds.
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: Byte offset 0, len 16384, i/o type 10.
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: If you get this error at mysqld startup, please check that
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: your my.cnf matches the ibdata files that you have in the
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: MySQL server.
Jan 10 02:35:14 jeepford mysqld[8400]: 200110 2:35:14InnoDB: Assertion failure in thread 140060547017040 in file fil0fil.c line 3959
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: We intentionally generate a memory trap.
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: If you get repeated assertion failures or crashes, even
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: immediately after the mysqld startup, there may be
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: corruption in the InnoDB tablespace. Please refer to
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
Jan 10 02:35:14 jeepford mysqld[8400]: InnoDB: about forcing recovery.
Jan 10 02:35:14 jeepford mysqld[8400]: 200110 2:35:14 - mysqld got signal 11 ;
Jan 10 02:35:14 jeepford mysqld[8400]: This could be because you hit a bug. It is also possible that this binary
Jan 10 02:35:14 jeepford mysqld[8400]: or one of the libraries it was linked against is corrupt, improperly built,
Jan 10 02:35:14 jeepford mysqld[8400]: or misconfigured. This error can also be caused by malfunctioning hardware.
Jan 10 02:35:14 jeepford mysqld[8400]: We will try our best to scrape up some info that will hopefully help diagnose
Jan 10 02:35:14 jeepford mysqld[8400]: the problem, but since we have already crashed, something is definitely wrong
Jan 10 02:35:14 jeepford mysqld[8400]: and this may fail.
Jan 10 02:35:14 jeepford mysqld[8400]:
Jan 10 02:35:14 jeepford mysqld[8400]: key_buffer_size=134217728
Jan 10 02:35:14 jeepford mysqld[8400]: read_buffer_size=6291456
Jan 10 02:35:14 jeepford mysqld[8400]: max_used_connections=0
Jan 10 02:35:14 jeepford mysqld[8400]: max_connections=200
Jan 10 02:35:14 jeepford mysqld[8400]: threads_connected=0
Jan 10 02:35:14 jeepford mysqld[8400]: It is possible that mysqld could use up to
Jan 10 02:35:14 jeepford mysqld[8400]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2588672 K
Jan 10 02:35:14 jeepford mysqld[8400]: bytes of memory
Jan 10 02:35:14 jeepford mysqld[8400]: Hope that's ok; if not, decrease some variables in the equation.
Jan 10 02:35:14 jeepford mysqld[8400]:
Jan 10 02:35:14 jeepford mysqld[8400]: thd=(nil)
Jan 10 02:35:14 jeepford mysqld[8400]: Attempting backtrace. You can use the following information to find out
Jan 10 02:35:14 jeepford mysqld[8400]: where mysqld died. If you see no messages after this, something went
Jan 10 02:35:14 jeepford mysqld[8400]: terribly wrong...
Jan 10 02:35:14 jeepford mysqld[8400]: frame pointer is NULL, did you compile with
Jan 10 02:35:14 jeepford mysqld[8400]: -fomit-frame-pointer? Aborting backtrace!
Jan 10 02:35:14 jeepford mysqld[8400]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Jan 10 02:35:14 jeepford mysqld[8400]: information that should help you find out what is causing the crash.
Jan 10 02:35:14 jeepford mysqld_safe[8436]: Number of processes running now: 0
Jan 10 02:35:14 jeepford mysqld_safe[8438]: restarted
Jan 10 02:35:14 jeepford mysqld[8441]: InnoDB: Log scan progressed past the checkpoint lsn 65 3180302167
Jan 10 02:35:14 jeepford mysqld[8441]: 200110 2:35:14 InnoDB: Database was not shut down normally!
Jan 10 02:35:14 jeepford mysqld[8441]: InnoDB: Starting crash recovery.
Jan 10 02:35:14 jeepford mysqld[8441]: InnoDB: Reading tablespace information from the .ibd files...
Jan 10 02:35:14 jeepford mysqld[8441]: InnoDB: Restoring possible half-written data pages from the doublewrite
Jan 10 02:35:14 jeepford mysqld[8441]: InnoDB: buffer...
Jan 10 02:35:14 jeepford mysqld[8441]: InnoDB: Doing recovery: scanned up to log sequence number 65 3185544704
Jan 10 02:35:15 jeepford mysqld[8441]: InnoDB: Doing recovery: scanned up to log sequence number 65 3186966404
Jan 10 02:35:15 jeepford mysqld[8441]: 200110 2:35:15 InnoDB: Starting an apply batch of log records to the database...
[/code]

gd_barnes 2020-01-10 08:45

Here are the first few pages after shutting machine down and turning it back on:

[code]
Jan 10 02:06:07 jeepford syslogd 1.5.0#5ubuntu3: restart.
Jan 10 02:06:07 jeepford anacron[3197]: Job `cron.daily' terminated (exit status: 1) (mailing output)
Jan 10 02:06:07 jeepford sendmail[4500]: 00A867GF004500: from=root, size=294, class=0, nrcpts=1, msgid=<202001100806.00A867GF004500@jeepford.no-ip.org>, relay=root@localhost
Jan 10 02:06:07 jeepford sm-mta[4501]: 00A867BR004501: from=<root@jeepford.no-ip.org>, size=570, class=0, nrcpts=1, msgid=<202001100806.00A867GF004500@jeepford.no-ip.org>, proto=ESMTP, daemon=MTA-v4, relay=jeepford.no-ip.org [127.0.0.1]
Jan 10 02:06:07 jeepford sendmail[4500]: 00A867GF004500: to=root, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30294, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (00A867BR004501 Message accepted for delivery)
Jan 10 02:06:07 jeepford anacron[3197]: Normal exit (1 job run)
Jan 10 02:06:07 jeepford sm-mta[4502]: 00A867BR004501: to=<root@jeepford.no-ip.org>, ctladdr=<root@jeepford.no-ip.org> (0/0), delay=00:00:00, xdelay=00:00:00, mailer=local, pri=30804, dsn=2.0.0, stat=Sent
Jan 10 02:06:08 jeepford avahi-daemon[3078]: Invalid query packet.
Jan 10 02:06:17 jeepford avahi-daemon[3078]: Invalid query packet.
Jan 10 02:06:19 jeepford mysqld[4344]: InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Jan 10 02:06:19 jeepford mysqld[4344]: InnoDB: Apply batch completed
Jan 10 02:06:19 jeepford mysqld[4344]: 200110 2:06:19 InnoDB: Started; log sequence number 65 3186966404
Jan 10 02:06:19 jeepford mysqld[4344]: 200110 2:06:19 [Note] /usr/sbin/mysqld: ready for connections.
Jan 10 02:06:19 jeepford mysqld[4344]: Version: '5.0.75-0ubuntu10.5' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: Error: trying to access page number 1768430057 in space 0,
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: space name ./ibdata1,
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: which is outside the tablespace bounds.
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: Byte offset 0, len 16384, i/o type 10.
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: If you get this error at mysqld startup, please check that
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: your my.cnf matches the ibdata files that you have in the
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: MySQL server.
Jan 10 02:06:20 jeepford mysqld[4344]: 200110 2:06:20InnoDB: Assertion failure in thread 140681679481168 in file fil0fil.c line 3959
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: We intentionally generate a memory trap.
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: If you get repeated assertion failures or crashes, even
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: immediately after the mysqld startup, there may be
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: corruption in the InnoDB tablespace. Please refer to
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
Jan 10 02:06:20 jeepford mysqld[4344]: InnoDB: about forcing recovery.
Jan 10 02:06:20 jeepford mysqld[4344]: 200110 2:06:20 - mysqld got signal 11 ;
Jan 10 02:06:20 jeepford mysqld[4344]: This could be because you hit a bug. It is also possible that this binary
Jan 10 02:06:20 jeepford mysqld[4344]: or one of the libraries it was linked against is corrupt, improperly built,
Jan 10 02:06:20 jeepford mysqld[4344]: or misconfigured. This error can also be caused by malfunctioning hardware.
Jan 10 02:06:20 jeepford mysqld[4344]: We will try our best to scrape up some info that will hopefully help diagnose
Jan 10 02:06:20 jeepford mysqld[4344]: the problem, but since we have already crashed, something is definitely wrong
Jan 10 02:06:20 jeepford mysqld[4344]: and this may fail.
Jan 10 02:06:20 jeepford mysqld[4344]:
Jan 10 02:06:20 jeepford mysqld[4344]: key_buffer_size=134217728
Jan 10 02:06:20 jeepford mysqld[4344]: read_buffer_size=6291456
Jan 10 02:06:20 jeepford mysqld[4344]: max_used_connections=0
Jan 10 02:06:20 jeepford mysqld[4344]: max_connections=200
Jan 10 02:06:20 jeepford mysqld[4344]: threads_connected=0
Jan 10 02:06:20 jeepford mysqld[4344]: It is possible that mysqld could use up to
Jan 10 02:06:20 jeepford mysqld[4344]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2588672 K
Jan 10 02:06:20 jeepford mysqld[4344]: bytes of memory
Jan 10 02:06:20 jeepford mysqld[4344]: Hope that's ok; if not, decrease some variables in the equation.
Jan 10 02:06:20 jeepford mysqld[4344]:
Jan 10 02:06:20 jeepford mysqld[4344]: thd=(nil)
Jan 10 02:06:20 jeepford mysqld[4344]: Attempting backtrace. You can use the following information to find out
Jan 10 02:06:20 jeepford mysqld[4344]: where mysqld died. If you see no messages after this, something went
Jan 10 02:06:20 jeepford mysqld[4344]: terribly wrong...
Jan 10 02:06:20 jeepford mysqld[4344]: frame pointer is NULL, did you compile with
Jan 10 02:06:20 jeepford mysqld[4344]: -fomit-frame-pointer? Aborting backtrace!
Jan 10 02:06:20 jeepford mysqld[4344]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Jan 10 02:06:20 jeepford mysqld[4344]: information that should help you find out what is causing the crash.
Jan 10 02:06:20 jeepford mysqld_safe[4515]: Number of processes running now: 0
Jan 10 02:06:20 jeepford mysqld_safe[4517]: restarted
Jan 10 02:06:20 jeepford mysqld[4520]: InnoDB: Log scan progressed past the checkpoint lsn 65 3180302167
Jan 10 02:06:20 jeepford mysqld[4520]: 200110 2:06:20 InnoDB: Database was not shut down normally!
Jan 10 02:06:20 jeepford mysqld[4520]: InnoDB: Starting crash recovery.
Jan 10 02:06:20 jeepford mysqld[4520]: InnoDB: Reading tablespace information from the .ibd files...
Jan 10 02:06:20 jeepford mysqld[4520]: InnoDB: Restoring possible half-written data pages from the doublewrite
Jan 10 02:06:20 jeepford mysqld[4520]: InnoDB: buffer...
Jan 10 02:06:21 jeepford mysqld[4520]: InnoDB: Doing recovery: scanned up to log sequence number 65 3185544704
Jan 10 02:06:21 jeepford mysqld[4520]: InnoDB: Doing recovery: scanned up to log sequence number 65 3186966404
Jan 10 02:06:21 jeepford mysqld[4520]: 200110 2:06:21 InnoDB: Starting an apply batch of log records to the database...
Jan 10 02:06:45 jeepford mysqld[4520]: InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Jan 10 02:06:45 jeepford mysqld[4520]: InnoDB: Apply batch completed
Jan 10 02:06:45 jeepford mysqld[4520]: 200110 2:06:45 InnoDB: Started; log sequence number 65 3186966404
Jan 10 02:06:45 jeepford mysqld[4520]: 200110 2:06:45 [Note] /usr/sbin/mysqld: ready for connections.
Jan 10 02:06:45 jeepford mysqld[4520]: Version: '5.0.75-0ubuntu10.5' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: Error: trying to access page number 1768430057 in space 0,
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: space name ./ibdata1,
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: which is outside the tablespace bounds.
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: Byte offset 0, len 16384, i/o type 10.
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: If you get this error at mysqld startup, please check that
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: your my.cnf matches the ibdata files that you have in the
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: MySQL server.
Jan 10 02:06:46 jeepford mysqld[4520]: 200110 2:06:46InnoDB: Assertion failure in thread 139678118078800 in file fil0fil.c line 3959
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: We intentionally generate a memory trap.
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: If you get repeated assertion failures or crashes, even
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: immediately after the mysqld startup, there may be
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: corruption in the InnoDB tablespace. Please refer to
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
Jan 10 02:06:46 jeepford mysqld[4520]: InnoDB: about forcing recovery.
Jan 10 02:06:46 jeepford mysqld[4520]: 200110 2:06:46 - mysqld got signal 11 ;
Jan 10 02:06:46 jeepford mysqld[4520]: This could be because you hit a bug. It is also possible that this binary
Jan 10 02:06:46 jeepford mysqld[4520]: or one of the libraries it was linked against is corrupt, improperly built,
Jan 10 02:06:46 jeepford mysqld[4520]: or misconfigured. This error can also be caused by malfunctioning hardware.
Jan 10 02:06:46 jeepford mysqld[4520]: We will try our best to scrape up some info that will hopefully help diagnose
Jan 10 02:06:46 jeepford mysqld[4520]: the problem, but since we have already crashed, something is definitely wrong
Jan 10 02:06:46 jeepford mysqld[4520]: and this may fail.
Jan 10 02:06:46 jeepford mysqld[4520]:
Jan 10 02:06:46 jeepford mysqld[4520]: key_buffer_size=134217728
Jan 10 02:06:46 jeepford mysqld[4520]: read_buffer_size=6291456
Jan 10 02:06:46 jeepford mysqld[4520]: max_used_connections=0
[/code]

rebirther 2020-01-10 09:54

try this here:
[url]https://chepri.com/mysql-innodb-corruption-and-recovery/[/url]

unconnected 2020-01-10 10:22

[URL]https://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html[/URL]
Or follow rebirther's link but start with innodb_force_recovery = 1
Recovering InnoDB databases isn't easy, possibly the simpliest way is to completely drop database and restore from latest backup if you have any.

gd_barnes 2020-01-10 21:02

I am trying Rebirther's suggested link. I was able to stop the SQL server in step 1. But I am unable to do steps 2 or 3 because I do not have permissions.

I get the following message:
You do not have the permissions necessary to open the file.

How do I get permissions to read, copy, write, or open these files?

I know it has something to do with having to be the root user but I cannot figure out how to do that.

One more thing: The my.cnf file is in /etc/mysql/my.cnf (not /etc/my.cnf).

rogue 2020-01-10 21:18

[QUOTE=gd_barnes;534805]I am trying Rebirther's suggested link. I was able to stop the SQL server in step 1. But I am unable to do steps 2 or 3 because I do not have permissions.

I get the following message:
You do not have the permissions necessary to open the file.

How do I get permissions to read, copy, write, or open these files?

I know it has something to do with having to be the root user but I cannot figure out how to do that.

One more thing: The my.cnf file is in /etc/mysql/my.cnf (not /etc/my.cnf).[/QUOTE]

Use "chown" or "chmod" to get access.

yoyo 2020-01-10 21:49

Yes, rebirther's link are the steps todo. Start with recovery=1 and check if the db starts.

You must be root, e.g. by "sudo bash" or "su -" to backup the IB files.

gd_barnes 2020-01-10 22:55

Using the link in Reb's post as a reference.

I was able to log in as root and backup the files in step #2.

Per step #3 I edited /etc/mysql/my.cnf to have the command:
innodb_force_recovery = 1

I'm now to step 4.

I stopped MySQL with:
sudo service mysql stop

Message said it stopped successfully.

I attempted to restart MySQL with:
sudo service mysql start

I got the following message:
* Starting MySQL database server mysqld [ OK ]
* Checking for corrupt, not cleanly closed and upgrade needing tables.
gary@jeepford:~$ ERROR 2013 (HY000) at line 2: Lost connection to MySQL server during query

This is the same message that I got before.

I then did an SQL status with this command:
sudo service mysql status

Output was this:
[code]
* /usr/bin/mysqladmin Ver 8.41 Distrib 5.0.75, for debian-linux-gnu on x86_64
Copyright (C) 2000-2006 MySQL AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license

Server version 5.0.75-0ubuntu10.5
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/run/mysqld/mysqld.sock
Uptime: 1 sec

Threads: 1 Questions: 2 Slow queries: 0 Opens: 12 Flush tables: 1 Open tables: 6 Queries per second avg: 2.000
[/code]My question is: Do I continue with step 5 by dumping all the tables? Or is the above error a problem that keeps me from continuing until it is fixed? I was hoping that the recovery line in my.cnf would correct this.

gd_barnes 2020-01-11 01:42

So...after doing steps 2 and 3 and running into that previous error on step 4...I thought I would try step 5 with the following command:

root@jeepford:/etc/mysql# mysqldump -u gary -p prpnet1300 > dump1300.sql

It then made me enter my root password, which I did. It then gave the following message:

mysqldump: Got error: 2002: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) when trying to connect

I cannot seem to get away from that message.

Note that I only tried to dump one of the servers. It would not allow me to use the -A command as specified in the instructions for #5.

Somehow we need to get it to correctly connect to the DB so that it can be dumped.

Help!

Happy5214 2020-01-11 04:26

I ran into this issue with my home server's previous OS install (Ubuntu 17.10 IIRC). I ended up reinstalling the OS (since I couldn't reinstall MySQL without killing my PIM setup), but this script may help get the server running so you can dump the contents (bypassing the permissions; make sure it's not publicly accessible):

[CODE]
#!/bin/sh

sudo mkdir /var/run/mysqld
sudo chown -R mysql /var/run/mysqld/
sudo mysqld --skip-grant-tables
[/CODE]

gd_barnes 2020-01-11 06:05

[QUOTE=Happy5214;534839]I ran into this issue with my home server's previous OS install (Ubuntu 17.10 IIRC). I ended up reinstalling the OS (since I couldn't reinstall MySQL without killing my PIM setup), but this script may help get the server running so you can dump the contents (bypassing the permissions; make sure it's not publicly accessible):

[CODE]
#!/bin/sh

sudo mkdir /var/run/mysqld
sudo chown -R mysql /var/run/mysqld/
sudo mysqld --skip-grant-tables
[/CODE][/QUOTE]

This did not work. I got the following gibberish:

[code]
gary@jeepford:~$ sudo mysqld --skip-grant-tables
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
200111 0:04:20 InnoDB: Retrying to lock the first data file
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
200111 0:04:33 InnoDB: ERROR: We were only able to scan the log up to
InnoDB: 65 3180301824, but a checkpoint was at 65 3180302287.
InnoDB: It is possible that the database is now corrupt!
200111 0:04:33 InnoDB: Error: page 1 log sequence number 65 3182452991
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 4 log sequence number 65 3181673642
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 6 log sequence number 65 3182821206
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 49172 log sequence number 65 3186685478
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 49153 log sequence number 65 3186345244
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 202272 log sequence number 65 3186682578
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 196609 log sequence number 65 3181146735
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 330293 log sequence number 65 3186631187
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 327681 log sequence number 65 3184275303
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 344075 log sequence number 65 3181039382
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 344065 log sequence number 65 3182245162
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 212993 log sequence number 65 3182730042
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 147457 log sequence number 65 3185010846
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 5 row operations to undo
InnoDB: Trx id counter is 0 2825215232
200111 0:04:33 InnoDB: Error: page 0 log sequence number 65 3182821206
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
InnoDB: Starting in background the rollback of uncommitted transactions
InnoDB: Cleaning up trx with id 0 2825225248
200111 0:04:33 InnoDB: Rollback of non-prepared transactions completed
200111 0:04:33 InnoDB: Error: page 311297 log sequence number 65 3183278764
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Error: page 114689 log sequence number 65 3181146898
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:33 InnoDB: Started; log sequence number 65 3180302287
InnoDB: !!! innodb_force_recovery is set to 1 !!!
200111 0:04:33 [Note] mysqld: ready for connections.
Version: '5.0.75-0ubuntu10.5' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
200111 0:04:34 InnoDB: Error: page 278529 log sequence number 65 3182822051
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 294913 log sequence number 65 3182821788
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 131073 log sequence number 65 3183083236
InnoDB: is in the future! Current system log sequence number 65 3180302287.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 65537 log sequence number 65 3181343415
InnoDB: is in the future! Current system log sequence number 65 3180302358.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 163841 log sequence number 65 3181146680
InnoDB: is in the future! Current system log sequence number 65 3180302358.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 229377 log sequence number 65 3181075900
InnoDB: is in the future! Current system log sequence number 65 3180302358.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 32769 log sequence number 65 3186349072
InnoDB: is in the future! Current system log sequence number 65 3180302358.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 192040 log sequence number 65 3180871058
InnoDB: is in the future! Current system log sequence number 65 3180302379.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 180225 log sequence number 65 3181147455
InnoDB: is in the future! Current system log sequence number 65 3180302379.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
200111 0:04:34 InnoDB: Error: page 43376 log sequence number 65 3181145698
InnoDB: is in the future! Current system log sequence number 65 3180302379.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
InnoDB: Error: trying to access page number 1768430057 in space 0,
InnoDB: space name ./ibdata1,
InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that
InnoDB: your my.cnf matches the ibdata files that you have in the
InnoDB: MySQL server.
200111 0:04:34InnoDB: Assertion failure in thread 139684868266320 in file fil0fil.c line 3959
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
200111 0:04:34 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=134217728
read_buffer_size=6291456
max_used_connections=0
max_connections=200
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2588672 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
frame pointer is NULL, did you compile with
-fomit-frame-pointer? Aborting backtrace!
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
[/code]

I tried it both with MySQL stopped and then again with it started. No luck.

yoyo 2020-01-11 06:59

First enable mysql logging. I have in [mysql]
log_error = /var/log/mysql/error.log

After you restarted mysql, check with e.g. "ps -ef | grep mysql" if mysql is running and with some sql command if you can access a table. You can also try to run an mysqldump. Check also the mysql error log. If mysql crashes you would see some segmentation fault of memory dumps.

If mysql is not running, set recovery=2 and try again.
And so on.
At least with recovery=5 mysql should start and report some error in error log.
But now you can run mysqldump.

gd_barnes 2020-01-11 10:17

With all of your help I've made quite a bit of progress.

I changed the recovery command to:
innodb_force_recovery = 5

That successfully started MySQL and allowed me to dump the databases.

When doing the dump with the -A command of course it dumps them all in one file. But I got a new database error message after a long time of dumping. I had a strong suspicion which database was corrupt on my machine. It is my personal server prpnet1470. Suspecting this I dumped only prpnet1470 and there it was with the same error message! All of the other databases dumped just fine.

Here is the message that I am getting for database prpnet1470 when I try to dump it:
mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `CandidateTestResult` at row: 251210

So if I follow Reb's link's instructions I'm going to be dumping all the databases, dropping them all, removing some files, restarting MySQL, and restoring all of the databases from the dump.

At this point, that seems like overkill.

My question is:
Should I just dump and drop prpnet1470? If so do I want to do step 8 which is: Remove /var/lib/mysql/ib*

I'm afraid to remove all of the ib* files from that library if I am dropping only one database.

I guess what I need is more specific instructions on fixing the one database that I know is corrupt, which is causing problems for MySQL.

I could even just permanently delete prpnet1470 and not restore it. I was able to do enough select statements to know where I'm at in my testing so there would be no data loss. It would be nice to keep it intact but if it is too much of a problem to do so there are other personal servers that I could use.

rebirther 2020-01-11 13:23

If you know the lib name of the affected database remove only the name of the lib file not all.

unconnected 2020-01-11 14:10

Gary, if you've successfully started mysql server, run mysql client and execute 'drop database prpnet1470' query in it.

MyDogBuster 2020-01-11 14:11

I'm going to rename this thread Gary's Great Adventure - All I ever wanted to know about MYSQL but were to afraid to ask

gd_barnes 2020-01-11 20:06

Haha. Good one Ian. I will move the applicable posts over to the "Server outrages" thread after the issue is completely resolved.


...trying the recent suggestions shortly.

gd_barnes 2020-01-11 20:57

I seem to be in a vicious loop here.

As previously specified I added the command:
innodb_force_recovery = 5
to the var/mysql/my.cnf file

I rebooted the machine and checked the MySQL status using:
sudo service mysql status

It looked good.

After logging into MySQL I tried to drop prpnet1470 with:
drop database prpnet1470;

I got the following error:
ERROR 2013 (HY000): Lost connection to MySQL server during query

So...MySQL isn't starting properly due to this error and I am unable to drop the offending DB due to this error.

Any suggestions?

I will try increasing the force_recovery even higher to see if that does anything.

gd_barnes 2020-01-11 21:03

So here is my latest attempt to start MySQL after changing my.cnf to:
innodb_force_recovery = 6

gary@jeepford:~$ sudo service mysql stop
* Stopping MySQL database server mysqld [ OK ]
gary@jeepford:~$ sudo service mysql start
* Starting MySQL database server mysqld [ OK ]
* Checking for corrupt, not cleanly closed and upgrade needing tables.

So...MySQL seems to start OK but is still giving this error message.

I'm stuck.

rebirther 2020-01-11 21:07

modify /etc/my.cnf with the following settings:



net_read_timeout=600
net_write_timeout=180
wait_timeout=86400
interactive_timeout=86400
max_allowed_packet=128M



Maybe the timeout is too short

gd_barnes 2020-01-11 21:17

I picked another random database to drop that is no longer used: prpnet1466

The database dropped just fine.

Trying Reb's suggestion next...

gd_barnes 2020-01-11 21:39

Adding the parameters in Reb's recent suggestion made no difference when attempting to drop database prpnet1470.

Maybe I could just manually give myself access to all of the root and mysql files where prpnet1470 is located and delete them all. Just a crazy thought.

rebirther 2020-01-11 21:51

[QUOTE=gd_barnes;534902]Adding the parameters in Reb's recent suggestion made no difference when attempting to drop database prpnet1470.

Maybe I could just manually give myself access to all of the root and mysql files where prpnet1470 is located and delete them all. Just a crazy thought.[/QUOTE]


How big is this database? If its bigger than max_allowed_packet=128M increase the value

gd_barnes 2020-01-12 00:10

[QUOTE=rebirther;534903]How big is this database? If its bigger than max_allowed_packet=128M increase the value[/QUOTE]

That's not the problem. I dumped one huge database that was 4.2 GB and it dumped fine.

Problem database prpnet1470 is about 200 MB so I increased the packet size to 256M, rebooted the machine, got MySQL up and running and tried to dump it. No difference.

Error:
Error 2013: Lost connection to MySQL server during query when dumping table `CandidateTestResult` at row: 251210

This error occurs after dumping 131.6 MB of the file.

gd_barnes 2020-01-12 00:14

A little more info:

I am able to do selects on all of the databases with the exception of the CandidateTestResult table on database prpnet1470. I can do selects on other tables in that one.

So no data will be lost except a small amount on prpnet1470.

But I cannot do any deletes on any table. My suspicion is that no table can be updated in any way...insert, update, or delete. That is why all of the prpnet servers start up just fine but when they try to receive a completed test an SQL error happens.

This begs the question: Why does one corrupt database affect MySQL in total such that other databases cannot be updated?

MyDogBuster 2020-01-12 02:04

Wild assed guess. I'm going to say that the MYSQL software is corrupt. Try installing
the MYSQL software again . I'm pretty sure that won't touch your data. Be sure the MYSQL service is stopped.

gd_barnes 2020-01-12 03:44

Servers are back up!

Thank you everyone for your help!

I was able to determine that several of the databases were corrupt so I dropped them all, recreated them, and reloaded them...all except my personal prpnet1470. All of the PRPnet servers work great now!

The problem was probably related to too many rows in prpnet1470 being hit by too many clients because that is where the problem started.

Bring on those machines! :smile:

I'm very sorry about the problem.

rebirther 2020-02-21 19:44

Hey Guys, now Iam affected after windows 10 has decided to give me a bluescreen, I cant take further actions, stucking at:


mysqldump: Got error: 1146: Table 'sr5.app' doesn't exist when using LOCK TABLES


Looks like a lot of tables are bad, enable skip lock tables doesnt help.


Any hints?


Update log:
[ERROR] Cannot find or open table sr5/app from
the internal data dictionary of InnoDB though the .frm file for the
table exists. Maybe you have deleted and recreated InnoDB data
files but have forgotten to delete the corresponding .frm files
of InnoDB tables, or you have moved .frm files to another database?
or, the table contains indexes that this version of the engine
doesn't support.


I think the files are lost.

dannyridel 2020-02-29 11:41

Again?
 
Is the server down again?
I can't upload my results to the server and my browser[I]s[/I] can't access the homepage.
The issue started about an hour ago.

rebirther 2020-03-01 17:19

[QUOTE=dannyridel;538575]Is the server down again?
I can't upload my results to the server and my browser[I]s[/I] can't access the homepage.
The issue started about an hour ago.[/QUOTE]


see news.

dannyridel 2020-03-03 03:53

A little suggestion, could you make the news send to the BOINC manager news section like PG does?
I don't look at it all day, and when I tried to return work it failed and got me alarmed looking at a "connection was reset" page.

gd_barnes 2020-03-05 12:23

I assume these recent posts are not related to NPLB's PRPnet servers, which appear to be working fine.

rebirther 2020-03-05 15:54

[QUOTE=gd_barnes;538941]I assume these recent posts are not related to NPLB's PRPnet servers, which appear to be working fine.[/QUOTE]



yes, you can delete these posts. My DB had not so much luck then yours :/

The Carnivore 2020-08-31 17:16

[url]http://www.noprimeleftbehind.net[/url] appears to be down at the time of this post.

gd_barnes 2020-08-31 19:37

Power outage this morning. Everything is back up.

gd_barnes 2021-04-27 16:18

It looks like the servers are down. I won't be back in town until late Saturday. I will work on them then. Sorry about the problem.

gd_barnes 2021-05-02 03:22

The servers are back up. Sorry about that.


All times are UTC. The time now is 19:47.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.