mersenneforum.org  

Go Back   mersenneforum.org > Extra Stuff > Programming

Reply
 
Thread Tools
Old 2022-05-10, 11:11   #1
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

2·19·41 Posts
Default Script for split file

I need script that will "ask" me number of workers and then split input file equally into worktodo.txt file ( and if possible add "Worker line") ( in this case 3 workers)


Quote:
example

input.txt
PRP=1,2,3
PRP=1,2,4
PRP=1,2,5


output

[Worker #1]
PRP=1,2,3
[Worker #2]
PRP=1,2,4
[Worker #3]
PRP=1,2,5

Thanks
pepi37 is online now   Reply With Quote
Old 2022-05-10, 11:17   #2
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

10000100111002 Posts
Default

Quote:
Originally Posted by pepi37 View Post
I need script that will "ask" me number of workers and then split input file equally into worktodo.txt file ( and if possible add "Worker line") ( in this case 3 workers)
Windows or Linux? What is your preferred (scripting) language?
paulunderwood is offline   Reply With Quote
Old 2022-05-10, 13:38   #3
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

43E16 Posts
Default

In bash:
Code:
if [ $# -ne 3 ];
then
  echo 'Usage: bash splitter.sh [in file] [out file] [number of workers]';
  exit -1;
fi;
if ls "$1."* 1> /dev/null 2>&1;
then
  echo 'Temporary files could not be created.';
  exit -2;
fi;
range=$(seq 1 $3);
if [ $? -ne 0 ];
then
  echo 'The worker count was not numerical.';
  exit -3;
fi;

for w in $range;
do
  echo "[Worker #$w]" >> "$1.tmp";
done;
cat "$1" >> "$1.tmp";
split -n r/$3 "$1.tmp" "$1.s";
cat "$1.s"* > "$2";
rm "$1."*;
kruoli is offline   Reply With Quote
Old 2022-05-10, 13:45   #4
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

110000101102 Posts
Default

Quote:
Originally Posted by paulunderwood View Post
Windows or Linux? What is your preferred (scripting) language?
Whatever I can run under Linux ๐Ÿ˜€
Language, everyone you like it. I will adopt
Thanks
pepi37 is online now   Reply With Quote
Old 2022-05-10, 13:54   #5
Uncwilly
6809 > 6502
 
Uncwilly's Avatar
 
"""""""""""""""""""
Aug 2003
101ร—103 Posts

2×13×409 Posts
Default

What about this tool? https://www.mersenne.ca/balance.php
Uncwilly is online now   Reply With Quote
Old 2022-05-10, 14:37   #6
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

2×19×41 Posts
Default

Quote:
Originally Posted by Uncwilly View Post
At first sight look perfect!
pepi37 is online now   Reply With Quote
Old 2022-05-10, 22:07   #7
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

2·19·41 Posts
Default

Quote:
Originally Posted by Uncwilly View Post
But doesnot work on PRP tests...
pepi37 is online now   Reply With Quote
Old 2022-05-10, 22:22   #8
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

2·19·41 Posts
Default

Quote:
Originally Posted by kruoli View Post
In bash:
Code:
if [ $# -ne 3 ];
then
  echo 'Usage: bash splitter.sh [in file] [out file] [number of workers]';
  exit -1;
fi;
if ls "$1."* 1> /dev/null 2>&1;
then
  echo 'Temporary files could not be created.';
  exit -2;
fi;
range=$(seq 1 $3);
if [ $? -ne 0 ];
then
  echo 'The worker count was not numerical.';
  exit -3;
fi;

for w in $range;
do
  echo "[Worker #$w]" >> "$1.tmp";
done;
cat "$1" >> "$1.tmp";
split -n r/$3 "$1.tmp" "$1.s";
cat "$1.s"* > "$2";
rm "$1."*;

Work perfect.
Now I will see can I adopt it to Win
Thanks!
pepi37 is online now   Reply With Quote
Old 2022-05-10, 22:27   #9
Uncwilly
6809 > 6502
 
Uncwilly's Avatar
 
"""""""""""""""""""
Aug 2003
101ร—103 Posts

2×13×409 Posts
Default

Quote:
Originally Posted by pepi37 View Post
But doesnot work on PRP tests...
I just tried it with some made up PRP lines and it recognized them, sorted them, and attempted to balance them.
Attached Thumbnails
Click image for larger version

Name:	works for me.png
Views:	17
Size:	81.4 KB
ID:	26859  
Uncwilly is online now   Reply With Quote
Old 2022-05-10, 22:53   #10
Batalov
 
Batalov's Avatar
 
"Serge"
Mar 2008
Phi(4,2^7658614+1)/2

990110 Posts
Default

bash
Code:
N=3
for i in `seq 1 $N`; do 
  echo '[Worker #'$i']'; awk 'a++%'$N'=='$i'-1' input.txt
done > output.txt
Batalov is offline   Reply With Quote
Old 2022-05-11, 00:37   #11
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

109C16 Posts
Default

Here is a perl script. usage: perl myscript.pl input_file worker_count > output_file

Code:
open(fh,"<",$ARGV[0]);
$count=0;
while(<fh>){$count++;}
close(fh);

open(fh,"<",$ARGV[0]);
$per_worker=int($count/$ARGV[1]);
$extras=$count%$ARGV[1];
$worker_blurb="[Worker \#";
$worker=0;
while(<fh>){
        $line=$_;
        $count++;
        if($count>$per_worker){
                $count=0;
                print("\n");
                $worker++;
                print($worker_blurb.$worker."]\n");
                if($extras==0){$per_worker--;}
                $extras--;
        }
        print($line);
}
close(fh);

Last fiddled with by paulunderwood on 2022-05-11 at 01:18
paulunderwood is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH script segment won't work in larger script EdH Programming 41 2021-10-15 22:07
CADO-NFS vs GGNFS and other NFS considerations [split from New Factoring Algorithm] Dubslow Factoring 6 2018-04-27 19:56
split a prime95 queue & client installation joblack Information & Answers 1 2009-01-06 08:45
Formula split up JHansen Homework Help 4 2007-11-20 15:02
New theorem to split LL tests(Published) nuggetprime Miscellaneous Math 9 2007-05-23 13:47

All times are UTC. The time now is 16:22.


Wed Aug 10 16:22:58 UTC 2022 up 34 days, 11:10, 3 users, load averages: 1.70, 1.44, 1.42

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

โ‰  ยฑ โˆ“ รท ร— ยท โˆ’ โˆš โ€ฐ โŠ— โŠ• โŠ– โŠ˜ โŠ™ โ‰ค โ‰ฅ โ‰ฆ โ‰ง โ‰จ โ‰ฉ โ‰บ โ‰ป โ‰ผ โ‰ฝ โŠ โŠ โŠ‘ โŠ’ ยฒ ยณ ยฐ
โˆ  โˆŸ ยฐ โ‰… ~ โ€– โŸ‚ โซ›
โ‰ก โ‰œ โ‰ˆ โˆ โˆž โ‰ช โ‰ซ โŒŠโŒ‹ โŒˆโŒ‰ โˆ˜ โˆ โˆ โˆ‘ โˆง โˆจ โˆฉ โˆช โจ€ โŠ• โŠ— ๐–• ๐–– ๐–— โŠฒ โŠณ
โˆ… โˆ– โˆ โ†ฆ โ†ฃ โˆฉ โˆช โŠ† โŠ‚ โŠ„ โŠŠ โŠ‡ โŠƒ โŠ… โŠ‹ โŠ– โˆˆ โˆ‰ โˆ‹ โˆŒ โ„• โ„ค โ„š โ„ โ„‚ โ„ต โ„ถ โ„ท โ„ธ ๐“Ÿ
ยฌ โˆจ โˆง โŠ• โ†’ โ† โ‡’ โ‡ โ‡” โˆ€ โˆƒ โˆ„ โˆด โˆต โŠค โŠฅ โŠข โŠจ โซค โŠฃ โ€ฆ โ‹ฏ โ‹ฎ โ‹ฐ โ‹ฑ
โˆซ โˆฌ โˆญ โˆฎ โˆฏ โˆฐ โˆ‡ โˆ† ฮด โˆ‚ โ„ฑ โ„’ โ„“
๐›ข๐›ผ ๐›ฃ๐›ฝ ๐›ค๐›พ ๐›ฅ๐›ฟ ๐›ฆ๐œ€๐œ– ๐›ง๐œ ๐›จ๐œ‚ ๐›ฉ๐œƒ๐œ— ๐›ช๐œ„ ๐›ซ๐œ… ๐›ฌ๐œ† ๐›ญ๐œ‡ ๐›ฎ๐œˆ ๐›ฏ๐œ‰ ๐›ฐ๐œŠ ๐›ฑ๐œ‹ ๐›ฒ๐œŒ ๐›ด๐œŽ๐œ ๐›ต๐œ ๐›ถ๐œ ๐›ท๐œ™๐œ‘ ๐›ธ๐œ’ ๐›น๐œ“ ๐›บ๐œ”