sm4(USP)

Name

sm4 : Inserts subtotal lines.

Synopsis

Usage   : sm4 <k1> <k2> <d1> <d2> [<s1> <s2>] <file>

          sm4 key=<key> [dummy=<dummy>] [val=<val>] <file>

Options : +<n>h

          -d<string>

          -e

Version : Tue Jan  9 09:02:34 JST 2024

Edition : 1

Description

Inserts a total line for records in <file> that have the same key.

Fields from <k1> to <k2> and are key fields while fields from <d1>

to <d2> are dummy fields (they are not keys nor are they summed).

Fields from <s1> to <s2> are the summed fields.  These fields are

summed for rows with the same key field and a total line is inserted

after the last line with the same key.  "@" characters are inserted

in the dummy fields of the total row.  If you use sm4 repeatedly to

insert subtotals and sub-subtotals then any record where there is at

least one field consists of only "@" will be excluded.  (Field that

includes "@" and other character such as "usp@usp-lab.com" are not

excluded.)

<key> designates the field position as fllows:

single field       2        the 2nd field

                   NF   the last field

                   NF-1   the field just before the last field

contiguous fields  2/4      from the 2nd field to the 4th field

                   4/2   from the 4th field to the 2nd field

                   NF-3/NF  from NF-3 field to the NF field

combination        2@NF     the 2nd field and the NF field

There is no limit on the length of the key field or on the number

of key fields.  The key field can also contain multi-byte characters

such as Japanese.

If you specify "r" as comparison method after the field position,

the fields are compared in reverse order.  If you specify "n" as

comparison method after the field position, that field's values will

be compared as numbers.  If you specify "nr" as comparison method

after the field, the values will be compared in reverse order as

numbers.  If you specify comparison method before or after the "/",

you must use the same comparison method for both fields.

  2n/5n     OK

  2n/5nr    Error

  2n/5r     Error

When you specify "e" as comparison method or specify -e ootion and no

method, characters in the field are replaced as follows and compared

as string:

  _  ==> 0x20 (space)

  \0 ==> 0x00 (null)

  \t ==> 0x09 (tab stop)

  \n ==> 0x0a (new line)

  \r ==> 0x0d (carrige return)

  \_ ==> 0x5f (underscore)

  \\ ==> 0x5c (back slash)

<val> is same as <key> except no comparison method is allowed.

The "+h" option treats the first row as a header row and excludes it

from the sum.

The "-d<string>" option changes the dummy string as <string>.  In

this case, the dummy string is inserted once for each dummy field

regardless the dummy field width.  Records which have a dummy field

equivalent to <string> is treated as total row.

Fields must be specified following these rules:

k1=1 k1<=k2 d1=k2+1 d1<=d2 s1=d2+1 s1<=s2 s2=NF(last field)

If you specify "x" for d1 or d2, then these rules must be followed:

k1=1 k1<=k2 s1=k2+1 s1<=s2 s2=NF(last field)

Example 1

Survey Data from Various Cities

$ fcols data

01 Massachusetts 01       Boston 91 59 20 76 54

01 Massachusetts 02    Worcester 46 39  8 5 21

01 Massachusetts 03  Springfield 82 0 23 84 10

02      New_York 04    Manhattan 30 50 71 36 30

02      New_York 05     Brooklyn 78 13 44 28 51

02      New_York 06       Queens 58 71 20 10 6

02      New_York 07       Albany 82 79 16 21 80

02      New_York 08      Buffalo 50  2 33 15 62

03    New_Jersey 09       Newark 52 91 44  9 0

03    New_Jersey 10      Trenton 60 89 33 18  6

03    New_Jersey 11   Moorestown 95 60 35 93 76

04  Pennsylvania 12 Philadelphia 92 56 83 96 75

04  Pennsylvania 13   Pittsburgh 30 12 32 44 19

04  Pennsylvania 14    Lancaster 48 66 23 71 24

Output subtotals by state:

$ sm4 1 2 3 4 5 NF data | fcols

01 Massachusetts 01       Boston  91 59 20 76 54

01 Massachusetts 02    Worcester  46 39   8 5 21

01 Massachusetts 03  Springfield 82   0 23 84 10

01 Massachusetts @@       @@@@@@ 219  98 51 165 85

02      New_York 04    Manhattan  30 50 71 36 30

02      New_York 05     Brooklyn  78 13 44 28 51

02      New_York 06       Queens 58 71 20 10   6

02      New_York 07       Albany 82 79 16 21 80

02      New_York 08      Buffalo  50   2 33 15 62

02      New_York @@       @@@@@@ 298 215 184 110 229

03    New_Jersey 09       Newark  52 91 44   9 0

03    New_Jersey 10      Trenton  60 89 33 18   6

03    New_Jersey 11   Moorestown  95 60 35 93 76

03    New_Jersey @@       @@@@@@ 207 240 112 120  82

04  Pennsylvania 12 Philadelphia 92 56 83 96 75

04  Pennsylvania 13   Pittsburgh 30 12 32 44 19

04  Pennsylvania 14    Lancaster 48 66 23 71 24

04  Pennsylvania @@       @@@@@@ 170 134 138 211 118

Example 2

Output subtotals and sub-subtotals.

$ fcols data2

01    Northeast 01 Massachusetts 01      Boston 91 59 20 76 54

01    Northeast 01 Massachusetts 03 Springfield 82  0 23 84 10

01    Northeast 02      New_York 04   Manhattan 30 50 71 36 30

01    Northeast 02      New_York 05    Brooklyn 78 13 44 28 51

01    Northeast 02      New_York 07      Albany 82 79 16 21 80

02 Mid-Atlantic 01    New_Jersey 01     Trenton 91 59 20 76 54

02 Mid-Atlantic 01    New_Jersey 02      Newark 46 39  8 5 21

02 Mid-Atlantic 01    New_Jersey 03  Moorestown 82 0 23 84 10

02 Mid-Atlantic 02      Virginia 04   Arlington 30 50 71 36 30

02 Mid-Atlantic 02      Virginia 05      Reston 78 13 44 28 51

02 Mid-Atlantic 02      Virginia 06     Langley 58 71 20 10  6

02 Mid-Atlantic 02      Virginia 07    Richmond 82 79 16 21 80

02 Mid-Atlantic 02      Virginia 08     Norfolk 50  2 33 15 62

02 Mid-Atlantic 03      Delaware 09       Dover 52 91 44 9 0

02 Mid-Atlantic 03      Delaware 10    Rehobeth 60 89 33 18  6

$ sm4 1 4 5 6 7 11 data2 | fcols

01    Northeast 01 Massachusetts 01      Boston  91 59 20 76 54

01    Northeast 01 Massachusetts 03 Springfield  82   0 23 84 10

01    Northeast 01 Massachusetts @@      @@@@@@ 173  59 43 160 64

01    Northeast 02      New_York 04   Manhattan  30 50 71 36 30

01    Northeast 02      New_York 05    Brooklyn  78 13 44 28 51

01    Northeast 02      New_York 07      Albany  82 79 16 21 80

01    Northeast 02      New_York @@      @@@@@@ 190 142 131  85 161

02 Mid-Atlantic 01    New_Jersey 01     Trenton  91 59 20 76 54

02 Mid-Atlantic 01    New_Jersey 02      Newark  46 39   8 5 21

02 Mid-Atlantic 01    New_Jersey 03  Moorestown 82   0 23 84 10

02 Mid-Atlantic 01    New_Jersey @@      @@@@@@ 219  98 51 165 85

02 Mid-Atlantic 02      Virginia 04   Arlington  30 50 71 36 30

02 Mid-Atlantic 02      Virginia 05      Reston  78 13 44 28 51

02 Mid-Atlantic 02      Virginia 06     Langley  58 71 20 10   6

02 Mid-Atlantic 02      Virginia 07    Richmond  82 79 16 21 80

02 Mid-Atlantic 02      Virginia 08     Norfolk  50   2 33 15 62

02 Mid-Atlantic 02      Virginia @@      @@@@@@ 298 215 184 110 229

02 Mid-Atlantic 03      Delaware 09       Dover 52 91 44   9 0

02 Mid-Atlantic 03      Delaware 10    Rehobeth  60 89 33 18   6

02 Mid-Atlantic 03      Delaware @@      @@@@@@ 112 180  77 27   6

$ sm4 1 4 5 6 7 11 data2 | sm4 1 2 3 6 7 11 | fcols

01    Northeast 01 Massachusetts 01      Boston  91 59 20 76 54

01    Northeast 01 Massachusetts 03 Springfield  82   0 23 84 10

01    Northeast 01 Massachusetts @@      @@@@@@ 173  59 43 160 64

01    Northeast 02      New_York 04   Manhattan  30 50 71 36 30

01    Northeast 02      New_York 05    Brooklyn  78 13 44 28 51

01    Northeast 02      New_York 07      Albany  82 79 16 21 80

01    Northeast 02      New_York @@      @@@@@@ 190 142 131  85 161

01    Northeast @@ @@@@@@@@@@@@@ @@      @@@@@@ 363 201 174 245 225

02 Mid-Atlantic 01    New_Jersey 01     Trenton  91 59 20 76 54

02 Mid-Atlantic 01    New_Jersey 02      Newark  46 39   8 5 21

02 Mid-Atlantic 01    New_Jersey 03  Moorestown 82   0 23 84 10

02 Mid-Atlantic 01    New_Jersey @@      @@@@@@ 219  98 51 165 85

02 Mid-Atlantic 02      Virginia 04   Arlington  30 50 71 36 30

02 Mid-Atlantic 02      Virginia 05      Reston  78 13 44 28 51

02 Mid-Atlantic 02      Virginia 06     Langley  58 71 20 10   6

02 Mid-Atlantic 02      Virginia 07    Richmond  82 79 16 21 80

02 Mid-Atlantic 02      Virginia 08     Norfolk  50   2 33 15 62

02 Mid-Atlantic 02      Virginia @@      @@@@@@ 298 215 184 110 229

02 Mid-Atlantic 03      Delaware 09       Dover 52 91 44   9 0

02 Mid-Atlantic 03      Delaware 10    Rehobeth  60 89 33 18   6

02 Mid-Atlantic 03      Delaware @@      @@@@@@ 112 180  77 27   6

02 Mid-Atlantic @@ @@@@@@@@@@@@@ @@      @@@@@@ 629 493 312 302 320

Example 3

Special Case (No Dummy Field)

$ fcols data3

Massachusetts 91 59 20 76 54

Massachusetts 46 39  8 5 21

Massachusetts 82  0 23 84 10

     New_York 30 50 71 36 30

     New_York 78 13 44 28 51

     New_York 58 71 20 10  6

     New_York 82 79 16 21 80

     New_York 50  2 33 15 62

   New_Jersey 52 91 44  9 0

   New_Jersey 60 89 33 18  6

   New_Jersey 95 60 35 93 76

 Pennsylvania 92 56 83 96 75

 Pennsylvania 30 12 32 44 19

 Pennsylvania 48 66 23 71 24

$ sm4 1 1 x x 2 6 data3 | fcols

Massachusetts  91  59 20 76 54

Massachusetts  46  39   8 5 21

Massachusetts  82   0  23 84 10

@@@@@@@@@@@@@ 219  98 51 165 85

     New_York  30  50 71 36 30

     New_York  78  13 44 28 51

     New_York  58  71 20 10   6

     New_York  82  79 16 21 80

     New_York  50   2  33 15 62

@@@@@@@@@@@@@ 298 215 184 110 229

   New_Jersey  52  91 44 9 0

   New_Jersey  60  89 33 18 6

   New_Jersey  95  60 35 93 76

@@@@@@@@@@@@@ 207 240 112 120  82

 Pennsylvania  92 56 83 96 75

 Pennsylvania  30 12 32 44 19

 Pennsylvania  48 66 23 71 24

@@@@@@@@@@@@@ 170 134 138 211 118

Example 4

-d<string> option

$ fcols data4

01 Massachusetts 01       Boston 91 59 20 76 54

01 Massachusetts 02    Worcester 46 39  8 5 21

01 Massachusetts 03  Springfield 82 0 23 84 10

02      New_York 04    Manhattan 30 50 71 36 30

02      New_York 05     Brooklyn 78 13 44 28 51

02      New_York 06       Queens 58 71 20 10 6

02      New_York 07       Albany 82 79 16 21 80

02      New_York 08      Buffalo 50  2 33 15 62

03    New_Jersey 09       Newark 52 91 44  9 0

03    New_Jersey 10      Trenton 60 89 33 18  6

03    New_Jersey 11   Moorestown 95 60 35 93 76

04  Pennsylvania 12 Philadelphia 92 56 83 96 75

04  Pennsylvania 13   Pittsburgh 30 12 32 44 19

04  Pennsylvania 14    Lancaster 48 66 23 71 24

$ sm4 -d+++ 1 2 3 4 5 NF data4 | fcols

01 Massachusetts  01       Boston 91 59 20 76 54

01 Massachusetts  02    Worcester 46 39   8 5 21

01 Massachusetts  03 Springfield 82   0 23 84 10

01 Massachusetts +++          +++ 219  98 51 165 85

02      New_York  04    Manhattan  30 50 71 36 30

02      New_York  05     Brooklyn  78 13 44 28 51

02      New_York  06       Queens 58 71 20 10   6

02      New_York  07       Albany 82 79 16 21 80

02      New_York  08      Buffalo 50   2 33 15 62

02      New_York +++          +++ 298 215 184 110 229

03    New_Jersey  09       Newark 52 91 44   9 0

03    New_Jersey  10      Trenton  60 89 33 18   6

03    New_Jersey  11   Moorestown  95 60 35 93 76

03    New_Jersey +++          +++ 207 240 112 120  82

04  Pennsylvania  12 Philadelphia 92 56 83 96 75

04  Pennsylvania  13   Pittsburgh 30 12 32 44 19

04  Pennsylvania  14    Lancaster 48 66 23 71 24

04  Pennsylvania +++          +++ 170 134 138 211 118