mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   PrimeNet (https://www.mersenneforum.org/forumdisplay.php?f=11)
-   -   OFFICIAL "SERVER PROBLEMS" THREAD (https://www.mersenneforum.org/showthread.php?t=5758)

TheMawn 2014-07-18 19:20

[QUOTE=Batalov;378495]It better go to the log, not to the viewers. Some may be compelled to spend hours checking their network documentation, and shaking random wires on their computer... :smile:[/QUOTE]

The green one is not the culprit. After twenty minutes of shaking I was only able to wreck my sound system but the website was still down.

chalsall 2014-07-18 20:22

[QUOTE=Prime95;378260]I have no idea. If I knew why the database crashes so often I might could answer that question more intelligently.[/QUOTE]

George: Silence from a leader is rarely welcome.

kracker 2014-07-18 20:55

[QUOTE=chalsall;378518]George: Silence from a leader is rarely welcome.[/QUOTE]

Not everyone can/does spend 10 hours a day in front of a monitor.

chalsall 2014-07-18 21:11

[QUOTE=kracker;378520]Not everyone can/does spend 10 hours a day in front of a monitor.[/QUOTE]

Agreed.

Those who can't should step down.

And, like George, I can't.

Prime95 2014-07-23 22:02

I found the IIS log files. Some random stats:

Server is handling about 14000 prime95 HTTP requests per hour. Each request is a CGI, PHP, and at least a few SQL statements.

Server is also handling about 3000 web requests per hour. This includes GPU72 spider requests. It counts individual CSS/GIF/JPEG/PNG files as well as PHP request that will fire off some SQL statements.

I'm not sure if that is a lot or a little or if that gets me any closer to finding bottlenecks.

chalsall 2014-07-24 19:16

[QUOTE=Prime95;378923]I'm not sure if that is a lot or a little or if that gets me any closer to finding bottlenecks.[/QUOTE]

From my perspective, the correlation seems to be a large number of "factor found" results being submitted. As an example, just after Oliver submits his weekly queue Primenet seems to stagger a bit.

There has been earlier discussion that the external program used to ensure a factor submitted (forgot its name) is actually a factor might be causing a memory (or some other resource) leak. Importantly, the fault might not be that of the called program, but could be that of the calling program and/or the OS.

I would be very happy to work with you, Scott and James to try to narrow this down. GPU72 isn't overly aggressive in its spidering (in my mind), but it could be reduced further for analysis purposes.

VictordeHolland 2014-07-25 14:34

She is down again, with SQL error code 08S01 - Communication link failure.

Uncwilly 2014-07-25 15:13

[QUOTE=VictordeHolland;379035]She is down again, with SQL error code 08S01 - Communication link failure.[/QUOTE]<aol>Me too!</aol> On vacation. Paying for wifi, where it used to be free.

lycorn 2014-07-25 15:23

[QUOTE=chalsall;378981]From my perspective, the correlation seems to be a large number of "factor found" results being submitted. As an example, just after Oliver submits his weekly queue Primenet seems to stagger a bit.

[/QUOTE]

That seems to be right. But much more than Oliver, we have TJAOI submitting thousands of small factors nearly every day. If there´s really a correlation between factors being checked in and Primenet glitches, it should be easily spotted following TJAOI submissions.

kladner 2014-07-25 16:43

[QUOTE=VictordeHolland;379035]She is down again, with SQL error code 08S01 - Communication link failure.[/QUOTE]

Same at 16:43 UTC.

lycorn 2014-07-25 16:46

Down here as well, with:

Warning: odbc_pconnect() [function.odbc-pconnect]: SQL error: [Microsoft][ODBC SQL Server Driver][Shared Memory]General network error. Check your network documentation., SQL state 08S01 in SQLConnect in C:\v5\www\2013\v5server\0.96_database.inc.php on line 21
pnErrorResult=3 pnErrorDetail=Database unavailable ==END==

Prime95 2014-07-25 18:24

I'm having difficulty logging in to restart the services. I'm pursuing other means of rebooting.

chalsall 2014-07-25 18:56

[QUOTE=Prime95;379060]I'm having difficulty logging in to restart the services. I'm pursuing other means of rebooting.[/QUOTE]

Please forgive me for being annoying, but, for example, 1and1, EC2 and Rackspace provide easy means of rebooting servers without being physically in front of them.

And, further, I earlier today lowered GPU72's spidering against Primenet. No real big downside -- clients for TF'ing and P-1'ing will still be able to get assignments. But their results won't be recognized until Primenet comes back on-line.

kladner 2014-07-25 19:05

17:05 UTC

It's Baaaaack!

Gordon 2014-07-25 19:50

[QUOTE=Prime95;378923]I found the IIS log files. Some random stats:

Server is handling about 14000 prime95 HTTP requests per hour. Each request is a CGI, PHP, and at least a few SQL statements.

Server is also handling about 3000 web requests per hour. This includes GPU72 spider requests. It counts individual CSS/GIF/JPEG/PNG files as well as PHP request that will fire off some SQL statements.

I'm not sure if that is a lot or a little or if that gets me any closer to finding bottlenecks.[/QUOTE]

So that's 5 i/o requests per second. Even if every one triggers 5 more that's still only 25.

A quick search for [URL="http://www.mssqltips.com/sqlservertip/2127/benchmarking-sql-server-io-with-sqlio/"]MS SQL server throughput[/URL] suggest even M$ server can manage writes of hundreds per second and reads of up to 15,000 per second.

Is this running on an old IBM-XT or something similar. Even a mobile phone can be a web server nowadays...

Madpoo 2014-07-26 02:23

[QUOTE=Gordon;379069]So that's 5 i/o requests per second. Even if every one triggers 5 more that's still only 25.

A quick search for [URL="http://www.mssqltips.com/sqlservertip/2127/benchmarking-sql-server-io-with-sqlio/"]MS SQL server throughput[/URL] suggest even M$ server can manage writes of hundreds per second and reads of up to 15,000 per second.

Is this running on an old IBM-XT or something similar. Even a mobile phone can be a web server nowadays...[/QUOTE]

Yeah, it doesn't seem like an excessive amount of activity, but I don't know what the hardware is like. I maintain a bunch of pretty active websites and the MS SQL backend can handle a LOT more IOPS on a system with dual X5550's and 24GB of RAM. The usual bottlenecks are disk subsystem... in our case the size of the database can eventually be cached entirely in that 24GB of RAM (with room to spare), so just getting the disks to read/write as fast as possible is the key. It's on an external storage unit, 12 x 144GB 15K SAS drives. Logs and data are using separate spindles which is another key point.

My guess is the same as others... it could be some delays in checking those "factor found" results... just a bottleneck as it checks a bunch that come in all at once? It should delay checking them or limit that to only half of the available CPU's or something so it doesn't entirely bog down the system.

Without re-architecting the whole process, the "simple" solution is to throw hardware at it. Works every time it's tried... find the bottleneck and instead of making it work better with what it has, just remove the bottleneck by beefing up that subsystem.

Prime95 2014-07-26 20:08

I'm taking the server down to see how long it takes to rebuild some indexes.

I freed up 4GB of server disk space by downloading log files to my computer. This should give me enough disk space to rebuild some indexes which are over 90% fragmented.

Unfortunately, indexes can only be rebuilt online with the Enterprise Edition of SQLServer.

chalsall 2014-07-26 20:40

[QUOTE=Prime95;379110]I'm taking the server down to see how long it takes to rebuild some indexes.[/QUOTE]

Coolness. :smile:

Please let any of us know if we can help in any way.

Prime95 2014-07-26 21:31

Back up.

I rebuilt the indexes on the "current assignments" and "available assignments" tables.

I learned some SQLServer kung fu in the process. In poking around, there are some indexes I doubt are ever used. There are some wasteful indexes (like a unique index on "exponent,factor" -- PHP already checks for this prior to doing an insert -- I'll bet that particular index is huge). There are some clustered indexes that should be nonclustered and there are some duplicate indexes.

I'm googling for a script that will yield a report on the entire database layout so that I can review it offline.

None of these improvements will "solve" our server problems, but they should make it a little better.

chalsall 2014-07-26 21:38

[QUOTE=Prime95;379112]None of these improvements will "solve" our server problems, but they should make it a little better.[/QUOTE]

Thanks, as always, for your work George.

But could you answer the questions which have already been asked (repeatedly)? For example, what is the hardware currently being used? Freeing up 4GB now-a-days is less than most systems' memory, let alone "swap", let alone storage....

Prime95 2014-07-27 00:36

[QUOTE=chalsall;379113]
But could you answer the questions which have already been asked (repeatedly)? For example, what is the hardware currently being used? [/QUOTE]

ProLiant DL360 G4p. 2GB memory. 2 SCSI 72GB disks in RAID configuration.

There is a second computer (specs I don't know) that acts as a daily dumping ground for the transaction logs.

pdazzl 2014-07-27 01:26

I would venture to say you just need to post a mailing address and watch donated memory and drives show up.

NBtarheel_33 2014-07-27 02:03

[QUOTE=Prime95;379123]ProLiant DL360 G4p. 2GB memory. 2 SCSI 72GB disks in RAID configuration.

There is a second computer (specs I don't know) that acts as a daily dumping ground for the transaction logs.[/QUOTE]

Wow, yeah, we *are* getting to the point where a smartphone would be more powerful.

Trying to surf the Internet or even run Windows or Office with 2GB of RAM would be quite a feat these days.

Prime95 2014-07-27 04:31

[QUOTE=pdazzl;379127]I would venture to say you just need to post a mailing address and watch donated memory and drives show up.[/QUOTE]

All useless without Scott's time to oversee transferring the databases to a new machine. BTW, we already have a volunteer offering a drives, RAM, or newer model.

pdazzl 2014-07-27 13:54

[QUOTE=Prime95;379135]All useless without Scott's time to oversee transferring the databases to a new machine. BTW, we already have a volunteer offering a drives, RAM, or newer model.[/QUOTE]

Is the data transfer happening in the near term? If not at least more ram couldn't hurt. Not sure they even sell pc's with 2gb ram anymore.

Prime95 2014-07-27 15:12

I deleted some redundant indexes last night. Overnight things didn't go well -- lots of timeouts.

My theory is some stored procedures were using the redundant indexes and are now using full table scans. I've marked all stored procedures for recompile.

I've temporarily disabled the hourly stats and reporting updates. If I get no timeouts over the next hour or two, I'll re-enable them.

Sorry for the inconvenience.

Madpoo 2014-07-27 16:59

[QUOTE=Prime95;379155]I deleted some redundant indexes last night. Overnight things didn't go well -- lots of timeouts.

My theory is some stored procedures were using the redundant indexes and are now using full table scans. I've marked all stored procedures for recompile.

I've temporarily disabled the hourly stats and reporting updates. If I get no timeouts over the next hour or two, I'll re-enable them.

Sorry for the inconvenience.[/QUOTE]

There are some handy little scripts you can use to see which sprocs execute the most often, which ones take the longest to run on each exec, etc. I use these from time to time on our production systems to catch any misbehaving (long executing) procs and target those for optimization (well, I let the developers enjoy that task, I just point at it).

If I'm starting out with something fresh that I'm not familiar with, I'd start by looking at which sprocs run the most often, and then see if they're executing in a decent amount of time. That's just the low hanging fruit... the ones that run the most need to run the fastest. :)

If there are some that take a super long time to run, but they don't run that often, they should probably be looked at but unless it's doing something bad like locking in a big select and taking forever, that's something to worry about later.

Here's a snippet that shows the execs of each sproc on some 'xxDATABASExx' (replace the DB name in the script) since SQL started:

SELECT DB_NAME(st.dbid) DBName
,OBJECT_SCHEMA_NAME(st.objectid,dbid) SchemaName
,OBJECT_NAME(st.objectid,dbid) StoredProcedure
,max(cp.usecounts) Execution_count
FROM sys.dm_exec_cached_plans cp
CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) st
where DB_NAME(st.dbid) is not null and cp.objtype = 'proc' and DB_NAME(st.dbid)='xxDATABASExx'
group by cp.plan_handle, DB_NAME(st.dbid),
OBJECT_SCHEMA_NAME(objectid,st.dbid),
OBJECT_NAME(objectid,st.dbid)
order by max(cp.usecounts) desc

Here's a script that shows execs sorted by their average exec time (again, replace 'xxDATABASExx' with the actual DB name of interest):

SELECT DB_NAME(st.dbid) DBName
,OBJECT_SCHEMA_NAME(objectid,st.dbid) SchemaName
,OBJECT_NAME(objectid,st.dbid) StoredProcedure
,max(cp.usecounts) execution_count
,sum(qs.total_elapsed_time) total_elapsed_time
,sum(qs.total_elapsed_time) / max(cp.usecounts) avg_elapsed_time
FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.plan_handle) st
join sys.dm_exec_cached_plans cp on qs.plan_handle = cp.plan_handle
where DB_NAME(st.dbid) is not null and cp.objtype = 'proc' and DB_NAME(st.dbid)='xxDATABASExx'
group by DB_NAME(st.dbid),OBJECT_SCHEMA_NAME(objectid,st.dbid), OBJECT_NAME(objectid,st.dbid)
order by avg_elapsed_time desc


Times are in milliseconds, FYI. I just looked at one of my production DB's... ugh, I see some nasty tasks that export lots of data that are taking as much as 95 minutes!! But it only ran once in the past few weeks, and it's full of NO LOCK so it's mostly safe. And the sproc that executes the most, with 53,603 of them, only takes 43 ms. :)

Anyway, those are some nice things to see if there are any sprocs that really need some extra attention and TLC. Target the ones that either run the most for optimization, and then target the ones that run often enough and are really slow.

Full disclosure, these aren't my scripts exactly, they're cut and pasted from some common SQL troubleshooting things with my own sorting and columns thrown in, so I won't take any credit (or blame) for them. :)

Prime95 2014-07-27 18:46

Hourly reports are back up. Looking good so far. Thanks for your patience while I learn more about SQLServer admin chores. Thanks madpoo for the queries - I will try them out.

Now up to almost 10GB free (about 14% of the disk). I need to do more index maintenance (clustering, redundant removal, rebuilding) but that will have to wait a few days.

TheMawn 2014-07-27 21:45

[QUOTE=NBtarheel_33;379128]Wow, yeah, we *are* getting to the point where a smartphone would be more powerful.

Trying to surf the Internet or even run Windows or Office with 2GB of RAM would be quite a feat these days.[/QUOTE]

Dual Core Xeons @ 3ish GHz with DDR2-800MHz memory. Yikes.

Well, there is something to be said about how things in the goode olde days were built. A 2014 server built on 2006 hardware and only now is showing its age.

Madpoo 2014-07-28 15:27

[QUOTE=TheMawn;379177]Dual Core Xeons @ 3ish GHz with DDR2-800MHz memory. Yikes.

Well, there is something to be said about how things in the goode olde days were built. A 2014 server built on 2006 hardware and only now is showing its age.[/QUOTE]

Those DL360 G4p machines were pretty good for their day. At my last gig I managed quite a few of those (and DL380's) and I'm sure some of them are probably still running there. They were pretty solid for sure, just getting long in the tooth.

chalsall 2014-07-28 18:05

[QUOTE=Prime95;379135]All useless without Scott's time to oversee transferring the databases to a new machine. BTW, we already have a volunteer offering a drives, RAM, or newer model.[/QUOTE]

Absolutely no disrespect intended, but why is Scott in the critical path?

Clearly things are not working well currently. Perhaps it's time for new modern hardware (which has been volunteered) and "new blood" (which has also been volunteered).

Just saying....

Madpoo 2014-07-28 22:16

[QUOTE=chalsall;379225]Absolutely no disrespect intended, but why is Scott in the critical path?

Clearly things are not working well currently. Perhaps it's time for new modern hardware (which has been volunteered) and "new blood" (which has also been volunteered).

Just saying....[/QUOTE]

I was curious myself and traced where the [url]www.mersenne.org[/url] server lives... it's in San Diego, where Scott hangs his hat too, hosted in an I2B Networks collocation. I'm presuming Scott pays the bill for that which is cool. I hope he gets donations because even a 1U server with power/network isn't all that cheap.

George is the guy who gets the fun of fussing with the server and tweaking it, but I'd guess that physical access to the system itself comes down to Scott.

Perhaps if Scott and George were amenable to it, an alternate location or at least an alternate contact in the San Diego area from out of the Primenet/GIMPS fans could be setup with access to I2B or to assist in server maintenance remotely, but I won't presume to suggest they *should*, just that I'm sure there are many folks willing to pitch in. After all, there could be times when Scott and/or George are on vacation or just plain unavailable to deal with server issues.

For example, I'm the only server admin in my team but when I take a vacation, I *have to* pass the baton to someone else in a similar role in another department... it's the anti-SPOF thing to do (SPOF = single point of failure... in my case, it's me :smile: ).

Batalov 2014-07-28 22:31

The infamous [URL="http://en.wikipedia.org/wiki/Bus_factor"]truck factor[/URL] ~= 2 here.

Madpoo 2014-07-28 23:03

[QUOTE=Batalov;379240]The infamous [URL="http://en.wikipedia.org/wiki/Bus_factor"]truck factor[/URL] ~= 2 here.[/QUOTE]

We used to joke about "what if you get hit by a bus", and then one day a couple years ago, one of the other admins here actually did get hit...well, not a truck or bus but a car, riding his bike to work. Banged up but okay, and yeah, those kinds of things get the management folks concerned about making sure knowledge is shared, roles are shared, etc.

Primenet isn't one of those things where if it were down for a while, money is being lost or productivity goes down the tubes, but I like the idea of backups for George and Scott. They're irreplaceable of course for all the work they've done and still do, but I say if George had someone taking care of the drudge work of maintaining a web/SQL server, he could focus like a laser on tweaking extra milliseconds of performance from the latest CPU/GPU.

Of course maybe he likes learning about SQL and the tuning that goes into it... learning new stuff can be fun. :smile:

Prime95 2014-07-29 01:35

[QUOTE=chalsall;379225]why is Scott in the critical path?[/QUOTE]

Madpoo got it right. Scott and the server are in San Diego. I'm in N.C.

Prime95 2014-07-29 01:46

[QUOTE=Madpoo;379242]Of course maybe he likes learning about SQL and the tuning that goes into it... learning new stuff can be fun. :smile:[/QUOTE]

Back in the day, I made a living knowing the ins and outs of SQL. Our company played a significant role in the development of ODBC. At the time, Sybase, SQLServer's predecessor, was among my least favorite databases.

Database admin is a different beast. Learning new stuff can be fun, the jury is still out as to whether learning SQLServer admin or later on Linux/MySql admin will be fun.

I agree that getting the truck factor above two is a good goal. I tried to shame James into becoming a backup, but apparently the spectre of learning Windows and SQLServer was too much for a sane human being to bear.

Madpoo 2014-07-29 02:44

Out of morbid curiosity (I do this on my own websites too), I did some performance analysis of the [url]www.mersenne.org[/url] homepage, and just from a web hosting point of view, there's probably some easy pickin's to get things going a little better.

1) There's a call to an image /images/SBSI_logo_2008.gif that's a large, ugly, animated GIF (153 KB). What's stranger, the link when clicked on is a holding page ("Web Page Under Construction"). Doesn't seem like the best use of 153 KB of download... over half of the total data on the home page. At least the server says it's cacheable for 1 week, but even that could be higher if it's really needed.

2) HTML isn't compressed by the server... just on the base page text/html, compressing it would take it from 26,363 bytes to around 8,661 or so. The CSS and JS could be compressed too. Given the server's vintage, using compression could increase the CPU usage too much, and I don't think IIS 5.0 (Windows 2000) has the same settings to tell it to stop compressing content if CPU usage gets too high. IIS will at least cache a compressed version of any static resources so it probably wouldn't really hurt, but hard to say.

3) The Google Analytics script isn't using the async script. Well, it's using "ga.js" which is the async version, but the way it's loaded isn't using the async method. It's an easy enough fix.

4) There's a counter being loaded from counter.digits.com that was frequently behaving pretty slow, at least when I tested. The counter at the bottom of pages is so 20th century anyway. :)

5) There are some other links at the bottom with images, "badges" I tihnk they're called, like Cool Site of the Day and Safe Surf. These are nothing more than old attempts at link boosting. If it were me I'd remove them, but if they really had to be there, make sure to add a nofollow so the page doesn't get penalized for link spamming.

6) There's really not enough static resources (images, scripts, css) to warrant a CDN, but thought I'd mention it.

7) The expirations of static items is just 1 week... if they're unlikely to be updated often, I recommend a 1-year expiration. If they ever do change, my strategy is to add version strings as a query parameter, like /2013/style.css?v=1.2, but then you need a system in place to make sure code is updated with new version strings whenever a new static resource is updated... if it's being built and deployed, that can be automated, or if updates are infrequent enough, doing it manually isn't a hassle as long as pages are templated. Even a search/replace to update the version strings is an easy approach.

8) I'm getting nitpicky, but the jquery.min.js is loading from the Google CDN. That's perfectly fine in the US, but I manage international websites and found that the Google CDN is horribly slow in Asia, Australia, South America, etc. Europe and North America they're about even, but if the audience is truly global, the Microsoft CDN is a better all around performer.

9) In general though, the page just loads slow because of back end work being done to load the page. On the home page it's not bad... there's some stuff in the left column showing daily stats ("Today's Numbers") but maybe that's only generated a few times per day, not in real time. But I was getting times of 1-2 seconds for the "time to first byte" which is pretty slow. That's my browser waiting for a response from the server.

Madpoo 2014-07-29 03:18

I almost missed one of the more obvious things... the Google Translate button up in the top right. :) Chrome offers to translate a page if it's not in your native language, so buttons like that are increasingly less useful I guess.

It doesn't hurt *too much* having it there, except it brings in it's own CSS (on an encrypted connection too) which might slow down the time when the page starts to render. At least it looked that way on my waterfall chart. I could be wrong. But that SSL connection to get the Google translate CSS... ick. SSL is great if you're passing data that should be encrypted, but Google's push to encrypt everything means slow initial connections... can't say I'm exactly on board, at least until SPDY / HTTP 2.0 gets further along, which can make up for those initial delays.

flagrantflowers 2014-07-29 03:32

[QUOTE=Madpoo;379268]…me offers to translate a page if it's not in your native language, so buttons like that are increasingly less useful I guess.

It doesn't hurt *too much* having it there…[/QUOTE]

All of that was pretty amazing, one of the better more educational posts I have ever seen on this board. I haven't been here long.

I would like to thank you for doing that if only for me to have the chance to read it.

Double plus good.

Mark Rose 2014-07-29 04:16

[QUOTE=Madpoo;379260]
2) HTML isn't compressed by the server... just on the base page text/html, compressing it would take it from 26,363 bytes to around 8,661 or so. The CSS and JS could be compressed too. Given the server's vintage, using compression could increase the CPU usage too much, and I don't think IIS 5.0 (Windows 2000) has the same settings to tell it to stop compressing content if CPU usage gets too high. IIS will at least cache a compressed version of any static resources so it probably wouldn't really hurt, but hard to say.
[/QUOTE]

gzip'ping text content, especially at the lower compression levels is very, very fast. It'll add less than a 1% overhead to the PHP scripts running on the server.

Another plus to using a CDN is to offload the simple requests from the server to lessen the amount of threads IIS will use. CloudFlare is free.

Madpoo 2014-07-29 04:41

[QUOTE=Mark Rose;379277]gzip'ping text content, especially at the lower compression levels is very, very fast. It'll add less than a 1% overhead to the PHP scripts running on the server.

Another plus to using a CDN is to offload the simple requests from the server to lessen the amount of threads IIS will use. CloudFlare is free.[/QUOTE]

Yeah, gzip at maybe level 2-3 is CPU friendly, I just wasn't sure if the current server is CPU bound and if that little bit would be a back breaking piece of straw.

I'm personally a big fan of CDN's... I think I looked at Cloudflare once upon a time but when it came down to it at my work, the only real option for us was Akamai because of our global presence. There are a lot of "okay" CDNs out there that have decent POPs in North America/Europe but in the rest of the world they really drag, or they piggy back their whole thing on EC2 in limited locations.

I didn't know Cloudflare had a free option but that could be pretty good for the handful of GIF/CSS/JS.

I'm kind of annoyed right now because I had 1 TB of data that we bought on VPS.net (which was actually using some other CDN) a couple years back, right before we switched to Akamai. Last time I checked my account was still there and I still had 1 TB of data "banked" up, but now my login isn't working. I thought I could donate that bandwidth but I guess not. Oh well... it was like $500 for 1 TB of transferred data so that kind of stinks, but we didn't use it for several years...it figures.

One thing did catch my eye when checking out the website... it's IIS 5 which equals Windows 2000. At least that's what the server puts in it's headers. If it's Windows 2000 it could very well be SQL 2000, so right away I'm thinking that just getting newer hardware *PLUS* updating the OS would be a nice bonus.

Maybe someone out there has an unused retail license of Windows/SQL. All I have are MSDN and VL licenses otherwise I'd be more than happy to donate a license to the cause. I should probably dig through my old pile of CDs...there could be a retail disc in there for 2008 versions maybe. Not for that current DL360 G4p though... Windows 2008/SQL 2008 would probably be worse than now on just 2GB of RAM and mirrored 72GB disk. :) It would work though... I've installed Windows 2008 on those systems. But they'd want more RAM/disk at least.

Mark Rose 2014-07-29 07:03

[QUOTE=Madpoo;379278]Yeah, gzip at maybe level 2-3 is CPU friendly, I just wasn't sure if the current server is CPU bound and if that little bit would be a back breaking piece of straw.[/quote]

Even at level 9 it's not that significant of overhead. We compress at 6 as much higher usually doesn't get much extra compression.

[quote]
I'm personally a big fan of CDN's... I think I looked at Cloudflare once upon a time but when it came down to it at my work, the only real option for us was Akamai because of our global presence. There are a lot of "okay" CDNs out there that have decent POPs in North America/Europe but in the rest of the world they really drag, or they piggy back their whole thing on EC2 in limited locations.
[/quote]

We use CloudFront. It has nowhere near the edges of Akamai, and nothing in Africa, but it works well enough for our purposes. They were willing to give us a significantly better price than Akamai.

[quote]
I didn't know Cloudflare had a free option but that could be pretty good for the handful of GIF/CSS/JS.
[/quote]

I've used it on small sites before. I would put the static resources on a different domain and let CloudFlare handle those, keeping the dynamic stuff served directly (along with robots.txt and favicon.ico). I don't think it's how they intend for their service to be used, but it works. :)

[quote]
I'm kind of annoyed right now because I had 1 TB of data that we bought on VPS.net (which was actually using some other CDN) a couple years back, right before we switched to Akamai. Last time I checked my account was still there and I still had 1 TB of data "banked" up, but now my login isn't working. I thought I could donate that bandwidth but I guess not. Oh well... it was like $500 for 1 TB of transferred data so that kind of stinks, but we didn't use it for several years...it figures.
[/quote]

Expensive traffic! Especially when it's not used!

[quote]
One thing did catch my eye when checking out the website... it's IIS 5 which equals Windows 2000. At least that's what the server puts in it's headers. If it's Windows 2000 it could very well be SQL 2000, so right away I'm thinking that just getting newer hardware *PLUS* updating the OS would be a nice bonus.

Maybe someone out there has an unused retail license of Windows/SQL. All I have are MSDN and VL licenses otherwise I'd be more than happy to donate a license to the cause. I should probably dig through my old pile of CDs...there could be a retail disc in there for 2008 versions maybe. Not for that current DL360 G4p though... Windows 2008/SQL 2008 would probably be worse than now on just 2GB of RAM and mirrored 72GB disk. :) It would work though... I've installed Windows 2008 on those systems. But they'd want more RAM/disk at least.[/QUOTE]

Windows 2000 is the last Windows I used regularly. :)

If Windows 2008 is anything like Vista, only 2 GB of RAM will have it swapping before it's finished booting lol

Brian-E 2014-07-29 09:36

[QUOTE=Madpoo;379268]I almost missed one of the more obvious things... the Google Translate button up in the top right. :) Chrome offers to translate a page if it's not in your native language, so buttons like that are increasingly less useful I guess.[/QUOTE]
I caution against underestimating the importance of a language translation facility on the page. Someone casually browsing the internet and coming across GIMPS by chance is more likely to sit up and take notice if the translation to their own language is available instantly without the need to bother to switch in other resources. It can mean the difference between taking the trouble to read about something and not doing so.

snme2pm1 2014-07-29 09:58

[QUOTE=Madpoo;379278]it's IIS 5 which equals Windows 2000[/QUOTE]

Such has been declared. at #212

[QUOTE=Prime95;374637]Um, Windows Server 2000. Isn't that the latest version?[/QUOTE]

Though some folk didn't quite recognise George's humor at the time.

A 32 bit system would impose limits on any notion of winding up memory.

Madpoo 2014-07-29 14:32

[QUOTE=snme2pm1;379295]A 32 bit system would impose limits on any notion of winding up memory.[/QUOTE]

Yeah, there's the rub. I thought at first maybe it was already running at least Windows 2003 @ 64-bit, but no. 4GB would be it but even then, the /3GB switch in the boot.ini and the overall wacky implementation still means you're at 3GB for user apps.

SQL 2000 databases can be migrated to SQL 2008 R2. Well, I should say, I think it has to be at least SQL 2000 SP4, but in theory it works most of the time.

IIS 5 stuff should run okay in 32-bit mode of IIS 7.5 (Windows 2008 R2). I imagine any ISAPI plugins would either work, or have IIS 7.5 compatible versions by now.

PHP stuff shouldn't be a problem, if that's the underlying stuff on the site.

If by any weird reason none of those would be upgradeable to newer stuff, it could be virtualized and segmented... P2V the server. Clone it so now there's 2 virtual machines and rename one or the other of them. Make one the SQL 2000 server (or maybe it is a newer SQL, I'm just guessing), and make the other one the web/ftp/app server. Then you're not cramming it all into one 4GB memory space at least.

Windows 2000 itself only supports 4 CPU's in the standard edition, so splitting it would also help, you could allocate 4 to each separate machine.

That wouldn't be a long term solution, but seems like a decent stopgap measure until a real migration so whatever else could take place.

Madpoo 2014-07-29 14:35

[QUOTE=Brian-E;379290]I caution against underestimating the importance of a language translation facility on the page. Someone casually browsing the internet and coming across GIMPS by chance is more likely to sit up and take notice if the translation to their own language is available instantly without the need to bother to switch in other resources. It can mean the difference between taking the trouble to read about something and not doing so.[/QUOTE]

You might be right. The site has Google Analytics on it... if it's instrumented right you could tell how many times someone actually uses the Google Translate button on the page.

Madpoo 2014-07-29 15:04

[QUOTE=Madpoo;379305]Windows 2000 itself only supports 4 CPU's in the standard edition, so splitting it would also help, you could allocate 4 to each separate machine.[/QUOTE]

I'll have to correct myself... Hyper-V in Windows 2008 R2 supports a Windows Server 2000 guest OS, but with only 1 virtual CPU. Go figure.

You might be looking at VMWare for SMP support on a Windows 2000 guest.

Seems more and more like just updating the OS itself is a good idea.

George: ZIP me up an image of the disk and I'll work on it. :smile:

NBtarheel_33 2014-07-30 06:17

[QUOTE=chalsall;379225]Absolutely no disrespect intended, but why is Scott in the critical path?

Clearly things are not working well currently. Perhaps it's time for new modern hardware (which has been volunteered) and "new blood" (which has also been volunteered).

Just saying....[/QUOTE]

There wouldn't even *be* a PrimeNet without Scott. His company Entropia was a pioneer in distributed/grid computing. We were a perfect example for what his services could do, and his services were just what we needed to coordinate GIMPS across an ever-expanding network of contributors.

AFAIK, Entropia has been subsumed by another company, and Scott is no longer involved with its operations, but he continued to be involved with PrimeNet. But he certainly deserves recognition as a big part of the GIMPS/PrimeNet we interact with each day.

NBtarheel_33 2014-07-30 06:30

[QUOTE=Madpoo;379260]

1) There's a call to an image /images/SBSI_logo_2008.gif that's a large, ugly, animated GIF (153 KB). What's stranger, the link when clicked on is a holding page ("Web Page Under Construction"). Doesn't seem like the best use of 153 KB of download... over half of the total data on the home page. At least the server says it's cacheable for 1 week, but even that could be higher if it's really needed.[/QUOTE]

That's the advertisement for Shaw Binary Services that appears on the left-hand side of (almost?) every PrimeNet page. Do they exist any more? Are they still providing free services to us? If no to either case, then there really is no purpose for the GIF.

[QUOTE=Madpoo]
4) There's a counter being loaded from counter.digits.com that was frequently behaving pretty slow, at least when I tested. The counter at the bottom of pages is so 20th century anyway. :)[/QUOTE]

Aww...but I have been waiting with bated breath for the last few years to see what happens after 9999999...:smile:

[QUOTE=Madpoo]5) There are some other links at the bottom with images, "badges" I tihnk they're called, like Cool Site of the Day and Safe Surf. These are nothing more than old attempts at link boosting. If it were me I'd remove them, but if they really had to be there, make sure to add a nofollow so the page doesn't get penalized for link spamming.[/QUOTE]

I bet the relevance of "Cool Site of the Day" and "Safe Surf" (whatever these might mean) both have long passed. If we are going to have badges/endorsements at all, it would be far more appropriate (and probably far better for our site ranking) to have official ones from e.g. the AMS, MAA, AMATYC, etc.

[QUOTE=Madpoo]9) In general though, the page just loads slow because of back end work being done to load the page. On the home page it's not bad... there's some stuff in the left column showing daily stats ("Today's Numbers") but maybe that's only generated a few times per day, not in real time. But I was getting times of 1-2 seconds for the "time to first byte" which is pretty slow. That's my browser waiting for a response from the server.[/QUOTE]

I believe the "Today's Numbers" are updated on the hour, along with the usual "standard" PrimeNet reports.

Batalov 2014-07-30 07:17

"In a few years, I might look at them and say..."
 
Those badges, strangely, remind me of an old Russian anecdote.
It is almost untranslatable, but I'll give it my best shot (I think I found a good equivalent of the punchline; without it, it is surely useless).

Needless to say, it is NSFW (as are most [URL="http://en.wikipedia.org/wiki/Russian_jokes#Poruchik_Rzhevsky"]Russian anecdotes[/URL]), so I will hide it.

[SPOILER]So, this guy dates an attractive young lady, and they are getting really close, you know. The lady blushes and goes: " I have to tell you something, so that this is not a shock to you.
I've been in love before. So much in love, in fact, that I tattooed his portrait on my left breast.
Well, to be perfectly honest, then, I've loved another person, too, and I loved him just as much, so I tattooed his portrait on my right breast."
The guy contemplates for a moment, and then suddenly breaks down laughing. She asks, "What's with you?"
The guy says: "In some twenty years, I might look at them and ask, 'Why the long faces?'"[/SPOILER]

KEP 2014-07-30 15:16

@Batalov: That was a really great translation... at least it made me laugh :smile:

chalsall 2014-07-30 16:05

[QUOTE=NBtarheel_33;379333]But he certainly deserves recognition as a big part of the GIMPS/PrimeNet we interact with each day.[/QUOTE]

Without question -- I never intended to imply otherwise. My apologies if it was take as such.

But, it is quite clear that what was implement before is not fairing so well now-a-days. Not really surprising considering the age of the current kit and software, and the ever increasing load on Primenet.

I'm in no way suggesting that Scott should be "ousted" -- simply that there is a lot of talent here willing to contribute time, hardware and/or money to either implement Primenet V6, and/or place it on stronger "iron". And, as raised above, there's the whole "bus" issue.

Brian-E 2014-07-30 17:38

The other side of the "bus issue" coin, I suppose, is the "too many cooks" issue. To avoid too many people contributing their own ideas and not working in harmony with each other, there would still need to be one person - or maybe two or three at the most if they have very similar ideas and are able to work [U]very[/U] closely with each other - co-ordinating it all. That would be a huge time investment for that one person or handful of people.

NBtarheel_33 2014-07-30 18:57

[QUOTE=Brian-E;379360]The other side of the "bus issue" coin, I suppose, is the "too many cooks" issue. To avoid too many people contributing their own ideas and not working in harmony with each other, there would still need to be one person - or maybe two or three at the most if they have very similar ideas and are able to work [U]very[/U] closely with each other - co-ordinating it all. That would be a huge time investment for that one person or handful of people.[/QUOTE]

I think this gets handled more via the judicious delegation of power/admin rights. Ideally, George (and Scott, and perhaps one other trusted person) would give the ultimate thumbs-up or -down to changes and new implementations that a "second-tier" group of say 4-8 people would be working on (and have just enough authority to do so without screwing up anything else).

Madpoo 2014-07-30 20:56

[QUOTE=NBtarheel_33;379333]There wouldn't even *be* a PrimeNet without Scott....[/QUOTE]

I was just thinking about when I started with GIMPS, the process involved emailing George to get going and he'd email out a list of exponents to work on. When you had some results, you emailed those back and would get more.

I remember when Primenet was starting out... I don't even remember if Prime95 supported it at first but you could sign up and get exponents and check in results through the web interface which was, now that I think about it, cutting edge stuff. When was that...1997-98?

Big strides since those times. Of course, here we are with the Primenet stuff running on Windows 2000, but I guess it's held up okay until the past year or so at any rate.

NBtarheel_33 2014-07-31 06:39

[QUOTE=Madpoo;379378]I was just thinking about when I started with GIMPS, the process involved emailing George to get going and he'd email out a list of exponents to work on. When you had some results, you emailed those back and would get more.

I remember when Primenet was starting out... I don't even remember if Prime95 supported it at first but you could sign up and get exponents and check in results through the web interface which was, now that I think about it, cutting edge stuff. When was that...1997-98?

Big strides since those times. Of course, here we are with the Primenet stuff running on Windows 2000, but I guess it's held up okay until the past year or so at any rate.[/QUOTE]

The amazing thing to me is that the project caught on like it did, and that neither George nor the participants (up to replacement of any attrition over the years) lost interest and left the project die out three, five, ten years later. It's extremely impressive for a project that began with a post on math.sci.net basically asking "who wants to look for Mersenne primes?".

chalsall 2014-07-31 18:57

[QUOTE=NBtarheel_33;379366]Ideally, George (and Scott, and perhaps one other trusted person) would give the ultimate thumbs-up or -down to changes and new implementations that a "second-tier" group of say 4-8 people would be working on (and have just enough authority to do so without screwing up anything else).[/QUOTE]

Indeed. And as James already has such trust and access, and a deep knowledge of PHP, (but, self admittedly, not of the WIMP stack) it would seem to make sense that he'd be the third authority for the server software.

The "outside developers" could contribute by way of Subversion or Git, working on (virtualized) test environments with representative (but not real) datasets. When the code is checked in, it is run through regression tests, and then approved (or not) by the three admins.

Three reasons for the test environments:

1. The first is obvious -- don't break the server!

2. Primenet relies on Assignment IDs (AIDs). If a hostile party has these, nasty things can be done (well, specifically, the assignment can be unreserved).

3. Primenet relies on matching residues. Very few should ever be able to see the residue of a first LL until a double-check has been completed.

The hardware supporting Primenet, and the sysadmin of same, is yet another dimension of this problem space.

Madpoo 2014-07-31 20:06

[QUOTE=chalsall;379445]...The hardware supporting Primenet, and the sysadmin of same, is yet another dimension of this problem space.[/QUOTE]

I've offered up a DL360 G5 server, but it sounds like it comes down to Scott's okay on it since he's the point person for physical access to the server itself. Just on the off-chance he wants it, I got the hardware together and installed Server 2012 on there... it could be a chore to migrate the website and databases to a new box/updated OS/SQL but the more I thought about Windows 2000/SQL 2000, there's really not much else that could happen, and it's an update that's long overdue IMHO.

Without knowing a single thing about the current web code or DB structure, I'd hazard a guess that it should port fairly well from IIS 5 to IIS 8.5, and from SQL 2000 to SQL 2008 R2/2012. It's unusual to find MSSQL stuff that doesn't upgrade well or has deprecated commands. The best way to find out is to try it.

I wouldn't mind helping with sys admin duties on any Windows based solution...that's my bread and butter, how I keep food in my belly and a roof over my head. Plus it's the kind of thing that really shouldn't take more than a few minutes a week really, once it's running. If it takes more than that to keep a multi-tier app going in a steady state, then something is wrong. :smile:

Mark Rose 2014-07-31 20:30

One possible issue with switching machines is the PHP version. There have been some small compatibility-breaking changes in every minor release, so it's possible that something may break.

Prime95 2014-07-31 20:52

Server is running IIS 3.0, PHP 5, SQLServer 2005.

Server is definitely not CPU-bound. I've done 15 minutes of research on gzip html. It seems like turning it on in IIS is a no-no. I might try turning it on in PHP instead.

James Heinrich 2014-07-31 21:23

[QUOTE=Mark Rose;379448]One possible issue with switching machines is the PHP version. There have been some small compatibility-breaking changes in every minor release, so it's possible that something may break.[/QUOTE][QUOTE=Prime95;379449]Server is running ... PHP 5[/QUOTE]There's not [i]many[/i] major compatibility issues across the minor versions, probably the most notable one being ereg* functions being deprecated (in favour of their preg* equivalent) in v5.3+. Another gotcha that I've seen crop up is the default characterset for htmlentities in v5.4+. For what it's worth, the current server is running PHP v5.2.5.

ET_ 2014-08-01 08:55

[QUOTE=James Heinrich;379453]There's not [i]many[/i] major compatibility issues across the minor versions, probably the most notable one being ereg* functions being deprecated (in favour of their preg* equivalent) in v5.3+. Another gotcha that I've seen crop up is the default characterset for htmlentities in v5.4+. For what it's worth, the current server is running PHP v5.2.5.[/QUOTE]

Here is a short list of incompatibilities between PHP 5.4 and 5.5:

[url]http://php.net/manual/en/migration55.incompatible.php[/url]

Rodrigo 2014-08-01 21:23

What a great idea: [URL]http://www.wfsb.com/story/26174225/police-do-not-call-us-if-facebook-is-down[/URL].

I'll have to remember that the next time that PrimeNet is down. :cmd: Where should I call -- San Diego police, or maybe the FBI? :wink:

Rodrigo

chalsall 2014-08-03 00:13

[QUOTE=Madpoo;379446]I've offered up a DL360 G5 server, but it sounds like it comes down to Scott's okay on it since he's the point person for physical access to the server itself. Just on the off-chance he wants it, I got the hardware together and installed Server 2012 on there...[/QUOTE]

George, sorry for being annoying, but as the saying goes, "the squeaky wheel gets the grease".

Any thoughts from you and Scott about Madpoo's very generous offer?

Prime95 2014-08-03 00:51

[QUOTE=chalsall;379579]George, sorry for being annoying, but as the saying goes, "the squeaky wheel gets the grease".

Any thoughts from you and Scott about Madpoo's very generous offer?[/QUOTE]

Madpoo and I are are actively working through a plan to migrate to his new server. Don't expect anything quick. We are looking at how to copy all the software, 25GB of data, and test it before shipping to San Diego. The goal is to be sure the transition is as easy as possible once the hardware arrives in its new home.

The only issue I can see that would scuttle this is whether madpoo has a valid donatable license to Windows and SQLServer. You'd think that would be trivial to figure out but MS licensing is a labyrinth.

Madpoo 2014-08-03 01:28

[QUOTE=Prime95;379583]The only issue I can see that would scuttle this is whether madpoo has a valid donatable license to Windows and SQLServer. You'd think that would be trivial to figure out but MS licensing is a labyrinth.[/QUOTE]

Yeah, so far I've gone back and looked at our old licenses for SQL/Windows and I'm coming up with a ton of old volume licenses, but those aren't transferable.

I'm pretty sure that at some point I had some retail licenses so I'm hoping as I dig through invoices for the past few years I'll find it.

If there is anyone out there who had a retail license of Windows 2008 or SQL 2008 (not a volume license or enterprise agreement) and then bought a new one for Server/SQL 2012, then that old license should be able to transfer, I think. As George said, MS licensing is kind of strange, but for transferring older products it's a little more straightforward, especially the retail packaged stuff.

They have non-profit pricing for MS products, FWIW.

I *do* have entire volume licensing agreements that were superseded... that is, a full VL suite for 2008 products (purchased in 2007) that we replaced with software assurance in 2008. I just don't know if those could be transferred... it's one of those things where it was cheaper to go to SA and basically abandon our investment of 50 server licenses (we'd only used 4 of them at the time).

I hate MS licensing, by the way. Like the products, but the licensing sucks. :smile:

Prime95 2014-08-03 01:57

I'm going to take the server down for some more maintenance. Clustering some indexes, some redundant index removal, and some other stuff. Cross your fingers.

TheMawn 2014-08-03 02:45

Does this mean the Bus factor is 2.01 and slowly rising to 3? That would be nice.

kladner 2014-08-03 04:27

Woo Hoo! It's Alive I tell you, Alive! 0427 UTC, 8/3/14

Prime95 2014-08-03 04:34

Yes, we're back. I completed worked on 6 tables -- clustering and rebuilding indexes. Removed one redundant index. Altered one column to NOT NULL (to save a couple of bytes in an index key).

I didn't get to 5 more tables. One redundant index on 20 million rows can be deleted. A few more columns can be changed to NOT NULL if there is enough disk space to create a transaction log.

The database has 9 GB free now -- but that cannot be turned into free disk space without destroying all that hard work defragging the indexes. There is 4GB free disk space, so we are not in bad shape.

kladner 2014-08-03 08:10

Many thanks! :tu:

chalsall 2014-08-05 15:58

[QUOTE=Madpoo;379585]If there is anyone out there who had a retail license of Windows 2008 or SQL 2008 (not a volume license or enterprise agreement) and then bought a new one for Server/SQL 2012, then that old license should be able to transfer, I think. As George said, MS licensing is kind of strange, but for transferring older products it's a little more straightforward, especially the retail packaged stuff.

[snip]

I hate MS licensing, by the way. Like the products, but the licensing sucks. :smile:[/QUOTE]

Just out of interest, could you give us all an idea as to how much it would cost for Mersenne Research, Inc. to purchase new licences of the required software?

And, to reflect, I personally hate the fact that the effectively zero cost to reproduce "bits" (read: proprietary software) often costs more than the hard to reproduce "atoms" (read: hardware).

And yet the latter probably took more money to develop, and definitely more to produce, and they have to get it right at release, rather than the "upgrade treadmill" of the former.

/end_rant

TheMawn 2014-08-05 16:57

[QUOTE=chalsall;379750]And, to reflect, I personally hate the fact that the effectively zero cost to reproduce "bits" (read: proprietary software) often costs more than the hard to reproduce "atoms" (read: hardware).

And yet the latter probably took more money to develop, and definitely more to produce, and they have to get it right at release, rather than the "upgrade treadmill" of the former.

/end_rant[/QUOTE]

I agree.

On the other hand, in the computer business, hardware gets made and sold on a larger scale than certain specialized programs. Still, I'm sure there's an aspect of greed where "we're the only ones who make something like this so we can safely make our customers pay through the nose."

My uncle works for the Canadian Government and they're still running Windows XP. He's got a head between his shoulders, but a lot of the people in his area don't and were pretty mad that XP support had stopped. Their reason for using Windows XP? The software they have licenses for costs $10,000 dollars per, and is not compatible with Windows 7, so they're sticking to XP.

My uncle actuall had to send a few dozen SSD's to be crushed and incinerated because the data on them can supposedly not be securely erased. They couldn't be given or sold to employees either because they already contained proprietary sensitive information. Otherwise, he said, he would have brought me as many SSD's as I had SATA ports because he knew I would appreciate them.

Said information was the fresh installation of Windows XP.


There's a bit of bureaucratic nonsense (I could tell stories about my dad at the car dealership, too) but it's surprising how much productivity is lost because of stuff like this. If the software was worth $1,000 per license instead, it would be cheap enough to warrant upgrading to the version with ten years worth of improvements and get a brand new computer to boot.

Madpoo 2014-08-05 18:03

[QUOTE=chalsall;379750]And yet the latter [HARDWARE] probably took more money to develop, and definitely more to produce, and they have to get it right at release, rather than the "upgrade treadmill" of the former.

/end_rant[/QUOTE]

Well... I deal with HP servers primarily and I can personally attest that they do NOT get it right at release, all too often. :)

It's rare, but happens, that I'll get a brand new system and through an actual design flaw, "something bad" is occurring. Most recently it was a new DL380p Gen8 that kept showing the array controller supercap was charging even though it was charged. Resulted in disabling the write caching since it thought it didn't have a good charge on it. Bummer. Firmware update fixed.

Look at any HP server at least and right after the system comes out, there'll be anywhere from 2-5 firmware updates per year for the first couple years, then tapering down from there.

A lot of those BIOS updates are to fix Intel errata... yeah, problems at the CPU level are a hoot and a holler, but thankfully they're usually minor. But cast ye minds back to 1994 and the Intel Pentium, with the infamous FDIV bug.
[url]http://en.wikipedia.org/wiki/Pentium_FDIV_bug[/url]

Okay, but to the larger point, yeah, software is buggy. :)

FWIW, many current and former HP customers are annoyed at a recent change... used to be you could get BIOS updates at no cost, in perpetuity, for any machines you have. Just go to their website, find the thing you want, and download. Just this year they introduced a fun new concept of only offering BIOS updates to customers with valid support contracts, or a system still under warranty. Wheee!

So imagine if FDIV bug 2.0 comes out on some new CPU and too bad, you don't have an HP support contract, you can't get a BIOS that corrects for Intel errata.

Yeah, some folks are none too happy. Technically I'm not bothered since we have contracts where I work, but I still sympathize. :smile:

Gordon 2014-08-05 22:41

[QUOTE=Prime95;379583]Madpoo and I are are actively working through a plan to migrate to his new server. Don't expect anything quick. We are looking at how to copy all the software, 25GB of data, and test it before shipping to San Diego. The goal is to be sure the transition is as easy as possible once the hardware arrives in its new home.

The only issue I can see that would scuttle this is whether madpoo has a valid donatable license to Windows and SQLServer. You'd think that would be trivial to figure out but MS licensing is a labyrinth.[/QUOTE]

Isn't this then the ideal time to ditch the proprietary software and switch to open-source?

If there is a "lot of work" involved in migrating it why not do the job properly?

Prime95 2014-08-05 22:51

[QUOTE=Gordon;379786]Isn't this then the ideal time to ditch the proprietary software and switch to open-source?

If there is a "lot of work" involved in migrating it why not do the job properly?[/QUOTE]

Yes, but there are two problems.

The short term problem is antiquated hardware running Windows 2000. This is most easily fixed by replacing the hardware with less antiquated hardware. The madpoo hardware is almost loaded up with data and ready for testing.

The long-term problem is coming up with a solution where GIMPS does not own the hardware. LAMP makes perfect sense but will take a while to research properly and implement.

Madpoo 2014-08-06 00:43

[QUOTE=Prime95;379789]Yes, but there are two problems.

The short term problem is antiquated hardware running Windows 2000. This is most easily fixed by replacing the hardware with less antiquated hardware. The madpoo hardware is almost loaded up with data and ready for testing.

The long-term problem is coming up with a solution where GIMPS does not own the hardware. LAMP makes perfect sense but will take a while to research properly and implement.[/QUOTE]

To what George is saying, it may seem like a baby step in terms of upgrading, but although Windows and MSSQL would still be the underlying bits and pieces, it's a pretty significant update in terms of performance, with relatively little effort (I think anyway) on the software side of things.

For testing things out, I setup a virtual machine that's roughly equivalent to the replacement hardware to see how all that code actually works when running under modern software. So far it seems like IIS 8.5 (Server 2012 R2) and the latest PHP (5.5) are doing fine. There's some database things to put together and try out (we're finally getting a backup of the mega database copied over right now) but the problems in dealing with an old system with very limited free drive space have been...interesting.

It's not that there's anything inherently wrong with the current setup from what I can tell, the hardware just hasn't kept up with the rising demands, plain and simple.

If we were to assume the end goal is to move to LAMP on a cloud-based machine, trust me, having all of this other migration done first will make that other migration much easier. :smile:

IIS -> Apache seems like it'd be a relatively simple thing in this case. There's nothing I saw that made me think it wouldn't migrate mostly cleanly. It's not .NET after all, it's PHP and unless there's anything Windows-centric, it seems like it'd be fine.

MSSQL -> MySQL would be the larger chore and George would know better than I since I've only had a very cursory look at the tables, sprocs, etc.

And hey, maybe George and y'all will love the new setup so much, we'll forget it's running Microsoft. :)

chalsall 2014-08-07 16:05

[QUOTE=Madpoo;379802]And hey, maybe George and y'all will love the new setup so much, we'll forget it's running Microsoft. :)[/QUOTE]

Just to say, I (and I expect, many) very much appreciate your very generous contribution of time, expertise and "kit" to our [STRIKE]addition[/STRIKE] hobby! :bow:

Mark Rose 2014-08-07 16:27

[QUOTE=chalsall;379933]our [STRIKE]addition[/STRIKE] hobby! :bow:[/QUOTE]

I thought it was more of a factoring than addition hobby ;)

Madpoo 2014-08-07 18:01

[QUOTE=chalsall;379933]Just to say, I (and I expect, many) very much appreciate your very generous contribution of time, expertise and "kit" to our [STRIKE]addition[/STRIKE] hobby! :bow:[/QUOTE]

It's fun for me... you know the saying, do what you love and you'll never work a day in your life.

George and I are still working through some migration things like making sure the msieve thing runs when a client reports a factor found (that's on my todo list for today), but so far things are going pretty well.

I don't want to steal George's thunder but let's just say that the test machine we're using right now (roughly the same as the replacement hardware, but on a VM that George can access) is MUCH faster. Like... a LOT.

I think at this point it's down to just a handful of environmental related settings needing attention, and then some good old fashioned, hardcore testing.

Mark Rose 2014-08-07 19:12

Will you be deploying inside of a VM on a new machine? That would make future hardware migrations easier.

Madpoo 2014-08-07 21:19

[QUOTE=Mark Rose;379951]Will you be deploying inside of a VM on a new machine? That would make future hardware migrations easier.[/QUOTE]

I've thought about it... Just straight Windows Hyper-V has very low overhead and I could copy over the VHDs of the test system and not worry about setting it all up again and matching all the tweaks.

And those VHD's can also be moved to a cloud server with little effort (whether it's Azure, Amazon, etc) besides the transfer of honkin' big VHD files itself. Or, as you say, future hardware migrations are simple as can be.

Having gone through years and years of painful hardware migrations, I'm a big fan of abstracting the purpose and function of a machine from the underlying hardware. Nobody wants to go through cloning drives, removing and installing drivers and whatever else. For a server, it makes total sense.

LaurV 2014-08-08 04:58

[QUOTE=Mark Rose;379935]I thought it was more of a factoring than addition hobby ;)[/QUOTE]
Hehe, nice one, this was more like those situation comedy, when you want to make a joke and another joke results instead... He wanted to write "addiction" most probably.

Madpoo 2014-08-08 07:43

[QUOTE=Madpoo;379942]George and I are still working through some migration things like making sure the msieve thing runs when a client reports a factor found (that's on my todo list for today), but so far things are going pretty well.[/QUOTE]

What I mean there was the "IsFactor" thing that runs, not MSieve.

Anyway... turns out that there are a set of 3 DLL's on the old server that are setup as extended procedures...

Unfortunately they are 32-bit DLL's which can't be called from 64-bit SQL. I have the source code but honestly, I'm no programmer. I've tried bringing them in to my Visual Studio 2013 and converting them to a more modern VS solution file... all of that goes fine, but I'm having trouble on the dependencies.

There's one regarding a opends60.lib file that's part of the SQL SDK ,so I grabbed that from 64-bit SQL but I seem to be banging my head against the wall with an error, like I'm just not linking it right.

Any C++ programmers out there want to take a crack at it and help compile three 64-bit DLLs? :smile:

Otherwise my only option is to setup a tiny SQL Express 32-bit version just to host these DLL's, called via a linked server connection on the same box. Kind of messy but it should work in theory.

James Heinrich 2014-08-08 14:27

Or if this is just needed to validate a submitted factor I can give you the cross-platform code that mersenne.ca uses to validate submitted factors. It runs on both Windows and *nix, and supports calling either PARI (and/or yafu on Windows) to validate the factor.

Madpoo 2014-08-08 15:39

[QUOTE=James Heinrich;380007]Or if this is just needed to validate a submitted factor I can give you the cross-platform code that mersenne.ca uses to validate submitted factors. It runs on both Windows and *nix, and supports calling either PARI (and/or yafu on Windows) to validate the factor.[/QUOTE]

If it helps, this is what the 3 extended procs are:

- "IsFactor" seems to just take the exponent and potential factor as arguments and they're passed into a UDF which just calls the extended proc without doing anything else. Returns 1 if the factor is correct, 0 if not, or -1 if there was an error

- "IsFactorOk" takes other arguments and it's also done on the site through a function that calls the XP. I'm ignorant enough to not know what it's checking, but it's called like this to see, as the comment in the code says, if the factor is P=1195 for example is consistent with 1.0, 2 and 1 in the arguments. :)
exec master..XP_IsFactorOK 1.0, 2, 1195, 1, '26537037220992112785174856161239437662001'

- "MSieve" is apparently a wrapper for Msieve factoring code ... pass in a number and it'll spit back 1 if prime, 0 if composite, or -1 if error. The UDF the site uses to call it has logic to call it in a loop, so the UDF there does make good sense as opposed to calling the XP directly.

I mention the use of UDF calling the XPs because... I went ahead and setup a 32-bit install of SQL Express and put the extended procs on there, and created a linked server. From the main SQL server I can call the remote XP just fine and it works. But if I adjust the UDF to call a remote function... SQL doesn't allow that. Says "Remote function calls are not allowed within a function."

So at this point I could use this hacky 32-bit thing and update the way the website checks by making it directly call the remote XP instead of using the UDF, but that wouldn't work so well for the msieve function with it's other block of logic...

I could also try turning those into sprocs instead of udfs which I think can do the remote call (haven't tried though).

My best bet would be getting these things compiled for 64-bit and call it good. I may take a fresh look today now that I'm rested... you know how it is when it's late and the code just won't make sense. :smile:

Prime95 2014-08-08 18:34

[QUOTE=James Heinrich;380007]Or if this is just needed to validate a submitted factor I can give you the cross-platform code that mersenne.ca uses to validate submitted factors. It runs on both Windows and *nix, and supports calling either PARI (and/or yafu on Windows) to validate the factor.[/QUOTE]


[strike]Is this portable code callable from PHP? If so, please send it in case we can't get the SQLServer function code working.[/strike] Nevermind: I've got a simple bc replacement.

Any ideas for a PHP interface to msieve (returning an array of factors)?

James Heinrich 2014-08-08 22:56

[QUOTE=Prime95;380020]Is this portable code callable from PHP? If so, please send it in case we can't get the SQLServer function code working.[/quote]It is. I actually already sent the code snippet to [i]Madpoo[/i], I can send it to you as well, although the unedited version is available [url=http://www.mersenne.ca/factorize.inc.php?showsource=1]here[/url] (this version does a quick check of small primes using my SQL table of all primes < 2[sup]32[/sup] but that part is easily deleted).

[QUOTE=Prime95;380020]Nevermind: I've got a simple bc replacement.[/quote]What do you mean by "bc replacement"? Do you mean like a PHP implementation using [url=http://ca1.php.net/manual/en/function.bcmul.php]bcmul[/url] and friends? I'd be curious to see how that compares to my own similar code (that I abandoned due to it being slow and using a ton of memory).

[QUOTE=Prime95;380020]Any ideas for a PHP interface to msieve (returning an array of factors)?[/QUOTE]My code will return an array of factors using PARI/GP. I haven't played with msieve but I suspect my code would be easily adapted to call that instead. I can look into it if you want.

Prime95 2014-08-08 23:22

[QUOTE=James Heinrich;380029]What do you mean by "bc replacement"? Do you mean like a PHP implementation using [url=http://ca1.php.net/manual/en/function.bcmul.php]bcmul[/url] and friends?[/quote]

Yes. I replaced the SQL extended proc that validates a reported factor actually divides the Mersenne number. Little more than a simple powmod and compare.


[quote]My code will return an array of factors using PARI/GP. I haven't played with msieve but I suspect my code would be easily adapted to call that instead. I can look into it if you want.[/QUOTE]

I got your yafu and pari examples. Thanks. This can replace the SQL extended proc that uses msieve to factor the occasional composite P-1 factor. Since I am not familiar with either PARI or YAFU, do you recommend one over the other?

Madpoo 2014-08-09 00:35

[QUOTE=Prime95;380030]Yes. I replaced the SQL extended proc that validates a reported factor actually divides the Mersenne number. Little more than a simple powmod and compare.

I got your yafu and pari examples. Thanks. This can replace the SQL extended proc that uses msieve to factor the occasional composite P-1 factor. Since I am not familiar with either PARI or YAFU, do you recommend one over the other?[/QUOTE]

I'm glad George is on to some alternatives... I got some schooling in Visual Studio beyond the normal plinking around I normally do for my job but wasn't any closer to getting it to compile in 64-bit. Some of that was due to other code that ties into it which would have to have been recompiled at 64-bit as well.

Part of all that was digging through old SQL sdks to get the old, unsupported, includes for extended procedures.

Doing these checks in a way that doesn't involve SQL at all is ideal, and really makes it that much more portable to future SQL editions, MySQL, or whatever.

NBtarheel_33 2014-08-09 01:28

With the implementation of PrimeNet v6, perhaps it might also be a good time to re-benchmark the GHz-day unit to a more modern CPU architecture.

Also, I would assume this probably will spell the end of the v4_computers, if indeed any are still contributing on a regular basis.

Prime95 2014-08-09 02:31

[QUOTE=James Heinrich;380029]My code will return an array of factors using PARI/GP. I haven't played with msieve but I suspect my code would be easily adapted to call that instead. I can look into it if you want.[/QUOTE]

I'm 99% there. Thanks, James. The only remaining issue is generating a unique filename for YAFU to write to instead of the fixed siqs.dat.

Prime95 2014-08-09 02:35

[QUOTE=NBtarheel_33;380036]With the implementation of PrimeNet v6, perhaps it might also be a good time to re-benchmark the GHz-day unit to a more modern CPU architecture.

Also, I would assume this probably will spell the end of the v4_computers, if indeed any are still contributing on a regular basis.[/QUOTE]

People fought replacing P95-years for a long, long time. I suspect they'd fight changing GHZ-days too.

I'm still getting several DCs a day from v4 computers. It turns out people are still downloading v24 prime95. Apparently, one can still create new v4 user IDs, get work, and report results. There is a PHP script that turns v4 requests into v5 requests, so there is no reason not to continue supporting v4 computers.

Madpoo 2014-08-09 02:49

[QUOTE=NBtarheel_33;380036]With the implementation of PrimeNet v6, perhaps it might also be a good time to re-benchmark the GHz-day unit to a more modern CPU architecture.

Also, I would assume this probably will spell the end of the v4_computers, if indeed any are still contributing on a regular basis.[/QUOTE]

Apparently there are still V4 results coming in on a daily basis... go figure. There's a PHP script that takes those old incoming calls and converts them or whatever to the correct V5 looking stuff.

That's actually one of the things we need to test out and make sure I set it up correctly.

I wouldn't call this upgrade V6, it's more like V5.5. The client/server communications are unchanged after all. :smile:

Madpoo 2014-08-09 02:51

[QUOTE=Prime95;380039]I'm 99% there. Thanks, James. The only remaining issue is generating a unique filename for YAFU to write to instead of the fixed siqs.dat.[/QUOTE]

I don't know how PHP does it, but a call to the system's GUID generator should ensure uniqueness. I'm just assuming PHP has something like that, otherwise a randomly generated one is probably just as good.

chalsall 2014-08-09 03:00

[QUOTE=Madpoo;380042]I'm just assuming PHP has something like that, otherwise a randomly generated one is probably just as good.[/QUOTE]

[url]http://php.net/manual/en/function.com-create-guid.php[/url]

James Heinrich 2014-08-09 04:08

[QUOTE=Prime95;380030]Since I am not familiar with either PARI or YAFU, do you recommend one over the other?[/QUOTE]I have PARI/GP working very nicely on both Windows and *nix. I can't say the same for either yafu or msieve (not saying it can't be done, I just haven't managed it).

My preference is for PARI/GP, it works well for me.


All times are UTC. The time now is 14:01.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.