View Single Post
Old 2020-12-22, 21:54   #17
jyb
 
jyb's Avatar
 
Aug 2005
Seattle, WA

110101011012 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
I haven't tried one of these big quartic jobs- if you could post memory use during filtering and the size of the resulting matrix, I'd appreciate it.
Code:
Tue Dec 22 09:26:03 2020  commencing relation filtering
Tue Dec 22 09:26:03 2020  setting target matrix density to 120.0
...
Tue Dec 22 10:03:52 2020  commencing duplicate removal, pass 2
Tue Dec 22 10:05:44 2020  found 0 duplicates and 376779347 unique relations
Tue Dec 22 10:05:44 2020  memory use: 1321.5 MB
Tue Dec 22 10:05:44 2020  reading ideals above 309919744
Tue Dec 22 10:05:44 2020  commencing singleton removal, initial pass
Tue Dec 22 10:31:57 2020  memory use: 6024.0 MB
Tue Dec 22 10:31:58 2020  reading all ideals from disk
Tue Dec 22 10:32:03 2020  memory use: 6904.4 MB
Tue Dec 22 10:32:19 2020  commencing in-memory singleton removal
Tue Dec 22 10:32:33 2020  begin with 376779347 relations and 310396245 unique ideals
Tue Dec 22 10:35:25 2020  reduce to 260546128 relations and 183665697 ideals in 14 passes
Tue Dec 22 10:35:25 2020  max relations containing the same ideal: 49
Tue Dec 22 10:35:44 2020  reading ideals above 720000
Tue Dec 22 10:35:45 2020  commencing singleton removal, initial pass
Tue Dec 22 11:01:01 2020  memory use: 6024.0 MB
Tue Dec 22 11:01:02 2020  reading all ideals from disk
Tue Dec 22 11:01:11 2020  memory use: 11439.4 MB
...
Tue Dec 22 12:11:07 2020  commencing 2-way merge
Tue Dec 22 12:11:58 2020  reduce to 55760982 relation sets and 54121813 unique ideals
Tue Dec 22 12:11:58 2020  commencing full merge
Tue Dec 22 12:35:29 2020  memory use: 7038.1 MB
...
Tue Dec 22 12:35:54 2020  commencing cycle optimization
Tue Dec 22 12:36:53 2020  start with 224975857 relations
Tue Dec 22 12:47:49 2020  pruned 10679506 relations
Tue Dec 22 12:47:49 2020  memory use: 5800.7 MB
...
Tue Dec 22 13:23:52 2020  matrix is 24785706 x 24785883 (11728.7 MB) with weight 3320871442 (133.98/col)
Tue Dec 22 13:23:52 2020  sparse part has weight 2801974712 (113.05/col)
Tue Dec 22 13:23:52 2020  saving the first 48 matrix rows for later
Tue Dec 22 13:23:56 2020  matrix includes 64 packed rows
Tue Dec 22 13:23:58 2020  matrix is 24785658 x 24785883 (11249.3 MB) with weight 2871892645 (115.87/col)
Tue Dec 22 13:23:58 2020  sparse part has weight 2701077747 (108.98/col)
Hopefully this gives you what you want to know. I'll post the full log when the job completes.
jyb is offline