gawk
Summary
This appendix provides a brief summary of the gawk
command line and the
awk
language. It is designed to serve as "quick reference." It is
therefore terse, but complete.
The command line consists of options to gawk
itself, the
awk
program text (if not supplied via the `-f' option), and
values to be made available in the ARGC
and ARGV
predefined awk
variables:
gawk [POSIX or GNU style options] -f source-file [--
] file ... gawk [POSIX or GNU style options] [--
] 'program' file ...
The options that gawk
accepts are:
-F fs
--field-separator fs
FS
predefined variable).
-f program-file
--file program-file
awk
program source from the file program-file, instead
of from the first command line argument.
-mf=NNN
-mr=NNN
gawk
, since gawk
has no predefined limits; they are only for compatibility with the
Bell Labs research version of Unix awk
.
-v var=val
--assign var=val
-W traditional
-W compat
--traditional
--compat
gawk
extensions are turned
off.
-W copyleft
-W copyright
--copyleft
--copyright
gawk
.
-W help
-W usage
--help
--usage
-W lint
--lint
awk
constructs.
-W lint-old
--lint-old
awk
.
-W posix
--posix
gawk
extensions
are turned off and additional restrictions apply.
-W re-interval
--re-interval
-W source=program-text
--source program-text
awk
program source code. This option allows
mixing command line source code with source code from files, and is
particularly useful for mixing command line programs with library functions.
-W version
--version
gawk
on the error
output.
--
awk
program itself to start with a `-'. This is mainly for
consistency with POSIX argument parsing conventions.
Any other options are flagged as invalid, but are otherwise ignored. See section Command Line Options, for more details.
An awk
program consists of a sequence of zero or more pattern-action
statements and optional function definitions. One or the other of the
pattern and action may be omitted.
pattern { action statements } pattern { action statements } function name(parameter list) { action statements }
gawk
first reads the program source from the
program-file(s), if specified, or from the first non-option
argument on the command line. The `-f' option may be used multiple
times on the command line. gawk
reads the program text from all
the program-file files, effectively concatenating them in the
order they are specified. This is useful for building libraries of
awk
functions, without having to include them in each new
awk
program that uses them. To use a library function in a file
from a program typed in on the command line, specify
`--source 'program'', and type your program in between the single
quotes.
See section Command Line Options.
The environment variable AWKPATH
specifies a search path to use
when finding source files named with the `-f' option. The default
path, which is
`.:/usr/local/share/awk'(23) is used if AWKPATH
is not set.
If a file name given to the `-f' option contains a `/' character,
no path search is performed.
See section The AWKPATH
Environment Variable.
gawk
compiles the program into an internal form, and then proceeds to
read each file named in the ARGV
array.
The initial values of ARGV
come from the command line arguments.
If there are no files named
on the command line, gawk
reads the standard input.
If a "file" named on the command line has the form `var=val', it is treated as a variable assignment: the variable var is assigned the value val. If any of the files have a value that is the null string, that element in the list is skipped.
For each record in the input, gawk
tests to see if it matches any
pattern in the awk
program. For each pattern that the record
matches, the associated action is executed.
awk
variables are not declared; they come into existence when they are
first used. Their values are either floating-point numbers or strings.
awk
also has one-dimensional arrays; multiple-dimensional arrays
may be simulated. There are several predefined variables that
awk
sets as a program runs; these are summarized below.
As each input line is read, gawk
splits the line into
fields, using the value of the FS
variable as the field
separator. If FS
is a single character, fields are separated by
that character. Otherwise, FS
is expected to be a full regular
expression. In the special case that FS
is a single space,
fields are separated by runs of spaces and/or tabs.
If FS
is the null string (""
), then each individual
character in the record becomes a separate field.
Note that the value
of IGNORECASE
(see section Case-sensitivity in Matching)
also affects how fields are split when FS
is a regular expression.
Each field in the input line may be referenced by its position, $1
,
$2
, and so on. $0
is the whole line. The value of a field may
be assigned to as well. Field numbers need not be constants:
n = 5 print $n
prints the fifth field in the input line. The variable NF
is set to
the total number of fields in the input line.
References to non-existent fields (i.e. fields after $NF
) return
the null string. However, assigning to a non-existent field (e.g.,
$(NF+2) = 5
) increases the value of NF
, creates any
intervening fields with the null string as their value, and causes the
value of $0
to be recomputed, with the fields being separated by
the value of OFS
.
See section Reading Input Files.
gawk
's built-in variables are:
ARGC
ARGV
. See below for what is actually
included in ARGV
.
ARGIND
ARGV
of the current file being processed.
When gawk
is processing the input data files,
it is always true that `FILENAME == ARGV[ARGIND]'.
ARGV
ARGC
- 1. Dynamically changing ARGC
and
the contents of ARGV
can control the files used for data. A null-valued element in
ARGV
is ignored. ARGV
does not include the options to
awk
or the text of the awk
program itself.
CONVFMT
FIELDWIDTHS
ENVIRON
HOME
is
ENVIRON["HOME"]
. One possible value might be `/home/arnold'.
Changing this array does not affect the environment seen by programs
which gawk
spawns via redirection or the system
function.
(This may change in a future version of gawk
.)
Some operating systems do not have environment variables.
The ENVIRON
array is empty when running on these systems.
ERRNO
getline
or close
.
FILENAME
FILENAME
is the null string.
FNR
FS
IGNORECASE
IGNORECASE
has a non-zero value, then pattern
matching in rules, record separating with RS
, field splitting
with FS
, regular expression matching with `~' and
`!~', and the gensub
, gsub
, index
,
match
, split
and sub
built-in functions all
ignore case when doing regular expression operations, and all string
comparisons are done ignoring case.
NF
NR
OFMT
print
statement,
"%.6g"
by default.
OFS
ORS
RS
RS
is set to the null string, then records are separated by
blank lines. When RS
is set to the null string, then the newline
character always acts as a field separator, in addition to whatever value
FS
may have. If RS
is set to a multi-character
string, it denotes a regexp; input text matching the regexp
separates records.
RT
RS
,
the record separator.
RSTART
match
; zero if no match.
RLENGTH
match
; -1 if no match.
SUBSEP
"\034"
.
See section Built-in Variables, for more information.
Arrays are subscripted with an expression between square brackets (`[' and `]'). Array subscripts are always strings; numbers are converted to strings as necessary, following the standard conversion rules (see section Conversion of Strings and Numbers).
If you use multiple expressions separated by commas inside the square
brackets, then the array subscript is a string consisting of the
concatenation of the individual subscript values, converted to strings,
separated by the subscript separator (the value of SUBSEP
).
The special operator in
may be used in a conditional context
to see if an array has an index consisting of a particular value.
if (val in array) print array[val]
If the array has multiple subscripts, use `(i, j, ...) in array' to test for existence of an element.
The in
construct may also be used in a for
loop to iterate
over all the elements of an array.
See section Scanning All Elements of an Array.
You can remove an element from an array using the delete
statement.
You can clear an entire array using `delete array'.
See section Arrays in awk
.
The value of an awk
expression is always either a number
or a string.
Some contexts (such as arithmetic operators) require numeric values. They convert strings to numbers by interpreting the text of the string as a number. If the string does not look like a number, it converts to zero.
Other contexts (such as concatenation) require string values.
They convert numbers to strings by effectively printing them
with sprintf
.
See section Conversion of Strings and Numbers, for the details.
To force conversion of a string value to a number, simply add zero to it. If the value you start with is already a number, this does not change it.
To force conversion of a numeric value to a string, concatenate it with the null string.
Comparisons are done numerically if both operands are numeric, or if
one is numeric and the other is a numeric string. Otherwise one or
both operands are converted to strings and a string comparison is
performed. Fields, getline
input, FILENAME
, ARGV
elements, ENVIRON
elements and the elements of an array created
by split
are the only items that can be numeric strings. String
constants, such as "3.1415927"
are not numeric strings, they are
string constants. The full rules for comparisons are described in
section Variable Typing and Comparison Expressions.
Uninitialized variables have the string value ""
(the null, or
empty, string). In contexts where a number is required, this is
equivalent to zero.
See section Variables, for more information on variable naming and initialization; see section Conversion of Strings and Numbers, for more information on how variable values are interpreted.
An awk
program is mostly composed of rules, each consisting of a
pattern followed by an action. The action is enclosed in `{' and
`}'. Either the pattern may be missing, or the action may be
missing, but not both. If the pattern is missing, the
action is executed for every input record. A missing action is
equivalent to `{ print }', which prints the entire line.
Comments begin with the `#' character, and continue until the end of the
line. Blank lines may be used to separate statements. Statements normally
end with a newline; however, this is not the case for lines ending in a
`,', `{', `?', `:', `&&', or `||'. Lines
ending in do
or else
also have their statements automatically
continued on the following line. In other cases, a line can be continued by
ending it with a `\', in which case the newline is ignored.
Multiple statements may be put on one line by separating each one with a `;'. This applies to both the statements within the action part of a rule (the usual case), and to the rule statements.
See section Comments in awk
Programs, for information on
awk
's commenting convention;
see section awk
Statements Versus Lines, for a
description of the line continuation mechanism in awk
.
awk
patterns may be one of the following:
/regular expression/ relational expression pattern && pattern pattern || pattern pattern ? pattern : pattern (pattern) ! pattern pattern1, pattern2 BEGIN END
BEGIN
and END
are two special kinds of patterns that are not
tested against the input. The action parts of all BEGIN
rules are
concatenated as if all the statements had been written in a single BEGIN
rule. They are executed before any of the input is read. Similarly, all the
END
rules are concatenated, and executed when all the input is exhausted (or
when an exit
statement is executed). BEGIN
and END
patterns cannot be combined with other patterns in pattern expressions.
BEGIN
and END
rules cannot have missing action parts.
For /regular-expression/
patterns, the associated statement is
executed for each input record that matches the regular expression. Regular
expressions are summarized below.
A relational expression may use any of the operators defined below in the section on actions. These generally test whether certain fields match certain regular expressions.
The `&&', `||', and `!' operators are logical "and," logical "or," and logical "not," respectively, as in C. They do short-circuit evaluation, also as in C, and are used for combining more primitive pattern expressions. As in most languages, parentheses may be used to change the order of evaluation.
The `?:' operator is like the same operator in C. If the first pattern matches, then the second pattern is matched against the input record; otherwise, the third is matched. Only one of the second and third patterns is matched.
The `pattern1, pattern2' form of a pattern is called a range pattern. It matches all input lines starting with a line that matches pattern1, and continuing until a line that matches pattern2, inclusive. A range pattern cannot be used as an operand of any of the pattern operators.
See section Pattern Elements.
Regular expressions are based on POSIX EREs (extended regular expressions). The escape sequences allowed in string constants are also valid in regular expressions (see section Escape Sequences). Regexps are composed of characters as follows:
c
\c
.
^
$
[abc...]
[[:class:]]
alnum
, alpha
, blank
, cntrl
,
digit
, graph
, lower
, print
, punct
,
space
, upper
, and xdigit
.
[[.symbol.]]
gawk
does not currently support collating symbols.
[[=chars=]]
gawk
does not currently support equivalence classes.
[^abc...]
r1|r2
r1r2
r+
r*
r?
(r)
r{n}
r{n,}
r{n,m}
\y
\B
\<
\>
\w
\W
\`
gawk
).
\'
The various command line options
control how gawk
interprets characters in regexps.
gawk
provide all the facilities of
POSIX regexps and the GNU regexp operators described above.
However, interval expressions are not supported.
--posix
--traditional
awk
regexps are matched. The GNU operators
are not special, interval expressions are not available, and neither
are the POSIX character classes ([[:alnum:]]
and so on).
Characters described by octal and hexadecimal escape sequences are
treated literally, even if they represent regexp metacharacters.
--re-interval
See section Regular Expressions.
Action statements are enclosed in braces, `{' and `}'. A missing action statement is equivalent to `{ print }'.
Action statements consist of the usual assignment, conditional, and looping statements found in most languages. The operators, control statements, and Input/Output statements available are similar to those in C.
Comments begin with the `#' character, and continue until the end of the
line. Blank lines may be used to separate statements. Statements normally
end with a newline; however, this is not the case for lines ending in a
`,', `{', `?', `:', `&&', or `||'. Lines
ending in do
or else
also have their statements automatically
continued on the following line. In other cases, a line can be continued by
ending it with a `\', in which case the newline is ignored.
Multiple statements may be put on one line by separating each one with a `;'. This applies to both the statements within the action part of a rule (the usual case), and to the rule statements.
See section Comments in awk
Programs, for information on
awk
's commenting convention;
see section awk
Statements Versus Lines, for a
description of the line continuation mechanism in awk
.
The operators in awk
, in order of decreasing precedence, are:
(...)
$
++ --
^
+ - !
* / %
+ -
space
< <= > >= != ==
~ !~
in
&&
||
?:
= += -= *= /= %= ^=
var=value
)
and operator assignment (the other forms) are supported.
See section Expressions.
The control statements are as follows:
if (condition) statement [ else statement ] while (condition) statement do statement while (condition) for (expr1; expr2; expr3) statement for (var in array) statement break continue delete array[index] delete array exit [ expression ] { statements }
See section Control Statements in Actions.
The Input/Output statements are as follows:
getline
$0
from next input record; set NF
, NR
, FNR
.
See section Explicit Input with getline
.
getline <file
$0
from next record of file; set NF
.
getline var
NF
, FNR
.
getline var <file
command | getline
getline
; sets $0
,
NF
, NR
.
command | getline var
getline
; sets var.
next
awk
program.
If the end of the input data is reached, the END
rule(s), if any,
are executed.
See section The next
Statement.
nextfile
FILENAME
is updated, FNR
is set to one,
ARGIND
is incremented,
and processing starts over with the first pattern in the awk
program.
If the end of the input data is reached, the END
rule(s), if any,
are executed.
Earlier versions of gawk
used `next file'; this usage is still
supported, but is considered to be deprecated.
See section The nextfile
Statement.
print
print expr-list
print expr-list > file
print
is executed.
print expr-list >> file
print
is appended to the file.
print expr-list | command
close
function
is called.
printf fmt, expr-list
printf fmt, expr-list > file
printf
is executed.
printf fmt, expr-list >> file
printf
is appended to the file.
printf fmt, expr-list | command
close
function
is called.
getline
returns zero on end of file, and -1 on an error.
In the event of an error, getline
will set ERRNO
to
the value of a system-dependent string that describes the error.
printf
Summary
Conversion specification have the form
%
[flag][width][.
prec]format.
Items in brackets are optional.
The awk
printf
statement and sprintf
function
accept the following conversion specification formats:
%c
%d
%i
%e
%E
%f
-
]ddd.dddddd
.
%g
%G
%o
%s
%x
%X
%%
There are optional, additional parameters that may lie between the `%' and the control letter:
-
space
+
#
0
width
.prec
Either or both of the width and prec values may be specified as `*'. In that case, the particular value is taken from the argument list.
See section Using printf
Statements for Fancier Printing.
When doing I/O redirection from either print
or printf
into a
file, or via getline
from a file, gawk
recognizes certain special
file names internally. These file names allow access to open file descriptors
inherited from gawk
's parent process (usually the shell). The
file names are:
In addition, reading the following files provides process related information
about the running gawk
program. All returned records are terminated
with a newline.
getuid
, geteuid
, getgid
, and getegid
system calls.
If there are any additional fields, they are the group IDs returned by
getgroups
system call.
(Multiple groups may not be supported on all systems.)
These file names may also be used on the command line to name data files. These file names are only recognized internally if you do not actually have files with these names on your system.
See section Special File Names in gawk
, for a longer description that
provides the motivation for this feature.
awk
provides a number of built-in functions for performing
numeric operations, string related operations, and I/O related operations.
The built-in arithmetic functions are:
atan2(y, x)
cos(expr)
exp(expr)
e ^ expr
).
int(expr)
log(expr)
expr
.
rand()
sin(expr)
sqrt(expr)
srand([expr])
awk
has the following built-in string functions:
gensub(regex, subst, how [, target])
$0
. The return value is the changed string; the
original target is not modified. Within subst,
`\n', where n is a digit from one to nine, can be used to
indicate the text that matched the n'th parenthesized
subexpression.
gsub(regex, subst [, target])
$0
.
index(str, search)
length([str])
$0
is returned if no argument is supplied.
match(str, regex)
RSTART
and RLENGTH
.
split(str, arr [, regex])
FS
is used instead. regex can be the null string, causing
each character to be placed into its own array element.
The array arr is cleared first.
sprintf(fmt, expr-list)
sub(regex, subst [, target])
gsub
, but only the first matching substring is replaced.
substr(str, index [, len])
tolower(str)
toupper(str)
The I/O related functions are:
close(expr)
fflush([expr])
""
), all output buffers are flushed.
system(cmd-line)
system
, calling it will
generate a fatal error.
`system("")' can be used to force awk
to flush any pending
output. This is more portable, but less obvious, than calling fflush
.
The following two functions are available for getting the current time of day, and for formatting time stamps.
systime()
strftime([format[, timestamp]])
date
utility is used if
no format is supplied.
See section Functions for Dealing with Time Stamps, for the
details on the conversion specifiers that strftime
accepts.
See section Built-in Functions, for a description of all of
awk
's built-in functions.
String constants in awk
are sequences of characters enclosed
in double quotes ("
). Within strings, certain escape sequences
are recognized, as in C. These are:
\\
\a
\b
\f
\n
\r
\t
\v
\xhex digits
"\x1B"
is a
string containing the ASCII ESC (escape) character. (The `\x'
escape sequence is not in POSIX awk
.)
\ddd
"\033"
is also a string containing the ASCII ESC
(escape) character.
\c
The escape sequences may also be used inside constant regular expressions
(e.g., the regexp /[ \t\f\n\r\v]/
matches whitespace
characters).
See section Escape Sequences.
Functions in awk
are defined as follows:
function name(parameter list) { statements }
Actual parameters supplied in the function call are used to instantiate the formal parameters declared in the function. Arrays are passed by reference, other variables are passed by value.
If there are fewer arguments passed than there are names in parameter-list, the extra names are given the null string as their value. Extra names have the effect of local variables.
The open-parenthesis in a function call of a user-defined function must immediately follow the function name, without any intervening white space. This is to avoid a syntactic ambiguity with the concatenation operator.
The word func
may be used in place of function
(but not in
POSIX awk
).
Use the return
statement to return a value from a function.
See section User-defined Functions.
There are two features of historical awk
implementations that
gawk
supports.
First, it is possible to call the length
built-in function not only
with no arguments, but even without parentheses!
a = length
is the same as either of
a = length() a = length($0)
For example:
$ echo abcdef | awk '{ print length }' -| 6
This feature is marked as "deprecated" in the POSIX standard, and
gawk
will issue a warning about its use if `--lint' is
specified on the command line.
(The ability to use length
this way was actually an accident of the
original Unix awk
implementation. If any built-in function used
$0
as its default argument, it was possible to call that function
without the parentheses. In particular, it was common practice to use
the length
function in this fashion, and this usage was documented
in the awk
manual page.)
The other historical feature is the use of either the break
statement,
or the continue
statement
outside the body of a while
, for
, or do
loop. Traditional
awk
implementations have treated such usage as equivalent to the
next
statement. More recent versions of Unix awk
do not allow
it. gawk
supports this usage if `--traditional' has been
specified.
See section Command Line Options, for more information about the `--posix' and `--lint' options.