Go to the previous, next section.
Expressions are the basic building block of awk
actions. An
expression evaluates to a value, which you can print, test, store in a
variable or pass to a function.
But, beyond that, an expression can assign a new value to a variable or a field, with an assignment operator.
An expression can serve as a statement on its own. Most other kinds of
statement contain one or more expressions which specify data to be
operated on. As in other languages, expressions in awk
include
variables, array references, constants, and function calls, as well as
combinations of these with various operators.
The simplest type of expression is the constant, which always has the same value. There are three types of constant: numeric constants, string constants, and regular expression constants.
A numeric constant stands for a number. This number can be an
integer, a decimal fraction, or a number in scientific (exponential)
notation. Note that all numeric values are represented within
awk
in double-precision floating point. Here are some examples
of numeric constants, which all have the same value:
105 1.05e+2 1050e-1
A string constant consists of a sequence of characters enclosed in double-quote marks. For example:
"parrot"
represents the string whose contents are `parrot'. Strings in
gawk
can be of any length and they can contain all the possible
8-bit ASCII characters including ASCII NUL. Other awk
implementations may have difficulty with some character codes.
Some characters cannot be included literally in a string constant. You represent them instead with escape sequences, which are character sequences beginning with a backslash (`\').
One use of an escape sequence is to include a double-quote character in
a string constant. Since a plain double-quote would end the string, you
must use `\"' to represent a single double-quote character as a
part of the string. Backslash itself is another character that can't be
included normally; you write `\\' to put one backslash in the
string. Thus, the string whose contents are the two characters
`"\' must be written "\"\\"
.
Another use of backslash is to represent unprintable characters such as newline. While there is nothing to stop you from writing most of these characters directly in a string constant, they may look ugly.
Here is a table of all the escape sequences used in awk
:
\\
\a
\b
\f
\n
\r
\t
\v
\nnn
\xhh...
A constant regexp is a regular expression description enclosed in
slashes, such as /^beginning and end$/
. Most regexps used in
awk
programs are constant, but the `~' and `!~'
operators can also match computed or "dynamic" regexps (see section How to Use Regular Expressions).
Constant regexps are useful only with the `~' and `!~' operators; you cannot assign them to variables or print them. They are not truly expressions in the usual sense.
Variables let you give names to values and refer to them later. You have
already seen variables in many of the examples. The name of a variable
must be a sequence of letters, digits and underscores, but it may not begin
with a digit. Case is significant in variable names; a
and A
are distinct variables.
A variable name is a valid expression by itself; it represents the variable's current value. Variables are given new values with assignment operators and increment operators. See section Assignment Expressions.
A few variables have special built-in meanings, such as FS
, the
field separator, and NF
, the number of fields in the current
input record. See section Built-in Variables, for a list of them. These
built-in variables can be used and assigned just like all other
variables, but their values are also used or changed automatically by
awk
. Each built-in variable's name is made entirely of upper case
letters.
Variables in awk
can be assigned either numeric values or string
values. By default, variables are initialized to the null string, which
is effectively zero if converted to a number. So there is no need to
"initialize" each variable explicitly in awk
, the way you would
need to do in C or most other traditional programming languages.
You can set any awk
variable by including a variable assignment
among the arguments on the command line when you invoke awk
(see section Invocation of awk
). Such an assignment has this form:
variable=text
With it, you can set a variable either at the beginning of the
awk
run or in between input files.
If you precede the assignment with the `-v' option, like this:
-v variable=text
then the variable is set at the very beginning, before even the
BEGIN
rules are run. The `-v' option and its assignment
must precede all the file name arguments.
Otherwise, the variable assignment is performed at a time determined by its position among the input file arguments: after the processing of the preceding input file argument. For example:
awk '{ print $n }' n=4 inventory-shipped n=2 BBS-list
prints the value of field number n
for all input records. Before
the first file is read, the command line sets the variable n
equal to 4. This causes the fourth field to be printed in lines from
the file `inventory-shipped'. After the first file has finished,
but before the second file is started, n
is set to 2, so that the
second field is printed in lines from `BBS-list'.
Command line arguments are made available for explicit examination by
the awk
program in an array named ARGV
(see section Built-in Variables).
The awk
language uses the common arithmetic operators when
evaluating expressions. All of these arithmetic operators follow normal
precedence rules, and work as you would expect them to. This example
divides field three by field four, adds field two, stores the result
into field one, and prints the resulting altered input record:
awk '{ $1 = $2 + $3 / $4; print }' inventory-shipped
The arithmetic operators in awk
are:
x + y
x - y
- x
x * y
x / y
awk
are double-precision
floating point, the result is not rounded to an integer: 3 / 4
has the value 0.75.
x % y
b * int(a / b) + (a % b) == a
One undesirable effect of this definition of remainder is that
x % y
is negative if x is negative. Thus,
-17 % 8 = -1
In other awk
implementations, the signedness of the remainder
may be machine dependent.
x ^ y
x ** y
2 ^ 3
has
the value 8. The character sequence `**' is equivalent to
`^'.
There is only one string operation: concatenation. It does not have a specific operator to represent it. Instead, concatenation is performed by writing expressions next to one another, with no operator. For example:
awk '{ print "Field number one: " $1 }' BBS-list
produces, for the first record in `BBS-list':
Field number one: aardvark
Without the space in the string constant after the `:', the line would run together. For example:
awk '{ print "Field number one:" $1 }' BBS-list
produces, for the first record in `BBS-list':
Field number one:aardvark
Since string concatenation does not have an explicit operator, it is
often necessary to insure that it happens where you want it to by
enclosing the items to be concatenated in parentheses. For example, the
following code fragment does not concatenate file
and name
as you might expect:
file = "file" name = "name" print "something meaningful" > file name
It is necessary to use the following:
print "something meaningful" > (file name)
We recommend you use parentheses around concatenation in all but the most common contexts (such as in the right-hand operand of `=').
Comparison expressions compare strings or numbers for relationships such as equality. They are written using relational operators, which are a superset of those in C. Here is a table of them:
x < y
x <= y
x > y
x >= y
x == y
x != y
x ~ y
x !~ y
subscript in array
Comparison expressions have the value 1 if true and 0 if false.
The operands of a relational operator are compared as numbers if they
are both numbers. Otherwise they are converted to, and compared as,
strings (see section Conversion of Strings and Numbers). Strings are compared by comparing the
first character of each, then the second character of each, and so on.
Thus, "10"
is less than "9"
.
For example,
$1 == "foo"
has the value of 1, or is true, if the first field of the current input record is precisely `foo'. By contrast,
$1 ~ /foo/
has the value 1 if the first field contains `foo'.
The right hand operand of the `~' and `!~' operators may be
either a constant regexp (/.../
), or it may be an ordinary
expression, in which case the value of the expression as a string is a
dynamic regexp (see section How to Use Regular Expressions).
In very recent implementations of awk
, a constant regular
expression in slashes by itself is also an expression. The regexp
/regexp/
is an abbreviation for this comparison expression:
$0 ~ /regexp/
In some contexts it may be necessary to write parentheses around the
regexp to avoid confusing the gawk
parser. For example,
(/x/ - /y/) > threshold
is not allowed, but ((/x/) - (/y/))
> threshold
parses properly.
One special place where /foo/
is not an abbreviation for
$0 ~ /foo/
is when it is the right-hand operand of `~' or
`!~'!
A boolean expression is combination of comparison expressions or matching expressions, using the boolean operators "or" (`||'), "and" (`&&'), and "not" (`!'), along with parentheses to control nesting. The truth of the boolean expression is computed by combining the truth values of the component expressions.
Boolean expressions can be used wherever comparison and matching
expressions can be used. They can be used in if
and while
statements. They have numeric values (1 if true, 0 if false), which
come into place if the result of the boolean expression is stored in a
variable, or used in arithmetic.
In addition, every boolean expression is also a valid boolean pattern, so you can use it as a pattern to control the execution of rules.
Here are descriptions of the three boolean operators, with an example of each. It may be instructive to compare these examples with the analogous examples of boolean patterns (see section Boolean Operators and Patterns), which use the same boolean operators in patterns instead of expressions.
boolean1 && boolean2
if ($0 ~ /2400/ && $0 ~ /foo/) print
The subexpression boolean2 is evaluated only if boolean1
is true. This can make a difference when boolean2 contains
expressions that have side effects: in the case of $0 ~ /foo/ &&
($2 == bar++)
, the variable bar
is not incremented if there is
no `foo' in the record.
boolean1 || boolean2
awk '{ if ($0 ~ /2400/ || $0 ~ /foo/) print }' BBS-list
The subexpression boolean2 is evaluated only if boolean1 is false. This can make a difference when boolean2 contains expressions that have side effects.
!boolean
awk '{ if (! ($0 ~ /foo/)) print }' BBS-list
An assignment is an expression that stores a new value into a
variable. For example, let's assign the value 1 to the variable
z
:
z = 1
After this expression is executed, the variable z
has the value 1.
Whatever old value z
had before the assignment is forgotten.
Assignments can store string values also. For example, this would store
the value "this food is good"
in the variable message
:
thing = "food" predicate = "good" message = "this " thing " is " predicate
(This also illustrates concatenation of strings.)
The `=' sign is called an assignment operator. It is the simplest assignment operator because the value of the right-hand operand is stored unchanged.
Most operators (addition, concatenation, and so on) have no effect except to compute a value. If you ignore the value, you might as well not use the operator. An assignment operator is different; it does produce a value, but even if you ignore the value, the assignment still makes itself felt through the alteration of the variable. We call this a side effect.
The left-hand operand of an assignment need not be a variable
(see section Variables); it can also be a field (see section Changing the Contents of a Field) or
an array element (see section Arrays in awk
). These are all called lvalues,
which means they can appear on the left-hand side of an assignment operator.
The right-hand operand may be any expression; it produces the new value
which the assignment stores in the specified variable, field or array
element.
It is important to note that variables do not have permanent types.
The type of a variable is simply the type of whatever value it happens
to hold at the moment. In the following program fragment, the variable
foo
has a numeric value at first, and a string value later on:
foo = 1 print foo foo = "bar" print foo
When the second assignment gives foo
a string value, the fact that
it previously had a numeric value is forgotten.
An assignment is an expression, so it has a value: the same value that
is assigned. Thus, z = 1
as an expression has the value 1.
One consequence of this is that you can write multiple assignments together:
x = y = z = 0
stores the value 0 in all three variables. It does this because the
value of z = 0
, which is 0, is stored into y
, and then
the value of y = z = 0
, which is 0, is stored into x
.
You can use an assignment anywhere an expression is called for. For
example, it is valid to write x != (y = 1)
to set y
to 1
and then test whether x
equals 1. But this style tends to make
programs hard to read; except in a one-shot program, you should
rewrite it to get rid of such nesting of assignments. This is never very
hard.
Aside from `=', there are several other assignment operators that
do arithmetic with the old value of the variable. For example, the
operator `+=' computes a new value by adding the right-hand value
to the old value of the variable. Thus, the following assignment adds
5 to the value of foo
:
foo += 5
This is precisely equivalent to the following:
foo = foo + 5
Use whichever one makes the meaning of your program clearer.
Here is a table of the arithmetic assignment operators. In each case, the right-hand operand is an expression whose value is converted to a number.
lvalue += increment
lvalue -= decrement
lvalue *= coefficient
lvalue /= quotient
lvalue %= modulus
lvalue ^= power
lvalue **= power
Increment operators increase or decrease the value of a variable
by 1. You could do the same thing with an assignment operator, so
the increment operators add no power to the awk
language; but they
are convenient abbreviations for something very common.
The operator to add 1 is written `++'. It can be used to increment a variable either before or after taking its value.
To pre-increment a variable v, write ++v
. This adds
1 to the value of v and that new value is also the value of this
expression. The assignment expression v += 1
is completely
equivalent.
Writing the `++' after the variable specifies post-increment. This
increments the variable value just the same; the difference is that the
value of the increment expression itself is the variable's old
value. Thus, if foo
has value 4, then the expression foo++
has the value 4, but it changes the value of foo
to 5.
The post-increment foo++
is nearly equivalent to writing (foo
+= 1) - 1
. It is not perfectly equivalent because all numbers in
awk
are floating point: in floating point, foo + 1 - 1
does
not necessarily equal foo
. But the difference is minute as
long as you stick to numbers that are fairly small (less than a trillion).
Any lvalue can be incremented. Fields and array elements are incremented just like variables.
The decrement operator `--' works just like `++' except that it subtracts 1 instead of adding. Like `++', it can be used before the lvalue to pre-decrement or after it to post-decrement.
Here is a summary of increment and decrement expressions.
++lvalue
lvalue++
--lvalue
++lvalue
, but instead of adding, it subtracts. It
decrements lvalue and delivers the value that results.
lvalue--
lvalue++
, but instead of adding, it subtracts. It
decrements lvalue. The value of the expression is the old
value of lvalue.
Strings are converted to numbers, and numbers to strings, if the context
of the awk
program demands it. For example, if the value of
either foo
or bar
in the expression foo + bar
happens to be a string, it is converted to a number before the addition
is performed. If numeric values appear in string concatenation, they
are converted to strings. Consider this:
two = 2; three = 3 print (two three) + 4
This eventually prints the (numeric) value 27. The numeric values of
the variables two
and three
are converted to strings and
concatenated together, and the resulting string is converted back to the
number 23, to which 4 is then added.
If, for some reason, you need to force a number to be converted to a string, concatenate the null string with that number. To force a string to be converted to a number, add zero to that string.
Strings are converted to numbers by interpreting them as numerals:
"2.5"
converts to 2.5, and "1e3"
converts to 1000.
Strings that can't be interpreted as valid numbers are converted to
zero.
The exact manner in which numbers are converted into strings is controlled
by the awk
built-in variable OFMT
(see section Built-in Variables).
Numbers are converted using a special
version of the sprintf
function (see section Built-in Functions) with OFMT
as the format specifier.
OFMT
's default value is "%.6g"
, which prints a value with
at least six significant digits. For some applications you will want to
change it to specify more precision. Double precision on most modern
machines gives you 16 or 17 decimal digits of precision.
Strange results can happen if you set OFMT
to a string that doesn't
tell sprintf
how to format floating point numbers in a useful way.
For example, if you forget the `%' in the format, all numbers will be
converted to the same constant string.
A conditional expression is a special kind of expression with three operands. It allows you to use one expression's value to select one of two other expressions.
The conditional expression looks the same as in the C language:
selector ? if-true-exp : if-false-exp
There are three subexpressions. The first, selector, is always computed first. If it is "true" (not zero) then if-true-exp is computed next and its value becomes the value of the whole expression. Otherwise, if-false-exp is computed next and its value becomes the value of the whole expression.
For example, this expression produces the absolute value of x
:
x > 0 ? x : -x
Each time the conditional expression is computed, exactly one of
if-true-exp and if-false-exp is computed; the other is ignored.
This is important when the expressions contain side effects. For example,
this conditional expression examines element i
of either array
a
or array b
, and increments i
.
x == y ? a[i++] : b[i++]
This is guaranteed to increment i
exactly once, because each time
one or the other of the two increment expressions is executed,
and the other is not.
A function is a name for a particular calculation. Because it has
a name, you can ask for it by name at any point in the program. For
example, the function sqrt
computes the square root of a number.
A fixed set of functions are built in, which means they are
available in every awk
program. The sqrt
function is one
of these. See section Built-in Functions, for a list of built-in functions and their
descriptions. In addition, you can define your own functions in the
program for use elsewhere in the same program. See section User-defined Functions,
for how to do this.
The way to use a function is with a function call expression, which consists of the function name followed by a list of arguments in parentheses. The arguments are expressions which give the raw materials for the calculation that the function will do. When there is more than one argument, they are separated by commas. If there are no arguments, write just `()' after the function name. Here are some examples:
sqrt(x**2 + y**2) # One argument atan2(y, x) # Two arguments rand() # No arguments
Do not put any space between the function name and the open-parenthesis! A user-defined function name looks just like the name of a variable, and space would make the expression look like concatenation of a variable with an expression inside parentheses. Space before the parenthesis is harmless with built-in functions, but it is best not to get into the habit of using space, lest you do likewise for a user-defined function one day by mistake.
Each function expects a particular number of arguments. For example, the
sqrt
function must be called with a single argument, the number
to take the square root of:
sqrt(argument)
Some of the built-in functions allow you to omit the final argument. If you do so, they use a reasonable default. See section Built-in Functions, for full details. If arguments are omitted in calls to user-defined functions, then those arguments are treated as local variables, initialized to the null string (see section User-defined Functions).
Like every other expression, the function call has a value, which is
computed by the function based on the arguments you give it. In this
example, the value of sqrt(argument)
is the square root of the
argument. A function can also have side effects, such as assigning the
values of certain variables or doing I/O.
Here is a command to read numbers, one number per line, and print the square root of each one:
awk '{ print "The square root of", $1, "is", sqrt($1) }'
Operator precedence determines how operators are grouped, when
different operators appear close by in one expression. For example,
`*' has higher precedence than `+'; thus, a + b * c
means to multiply b
and c
, and then add a
to the
product.
You can overrule the precedence of the operators by writing parentheses yourself. You can think of the precedence rules as saying where the parentheses are assumed if you do not write parentheses yourself. In fact, it is wise always to use parentheses whenever you have an unusual combination of operators, because other people who read the program may not remember what the precedence is in this case. You might forget, too; then you could make a mistake. Explicit parentheses will prevent any such mistake.
When operators of equal precedence are used together, the leftmost
operator groups first, except for the assignment, conditional and
and exponentiation operators, which group in the opposite order.
Thus, a - b + c
groups as (a - b) + c
;
a = b = c
groups as a = (b = c)
.
The precedence of prefix unary operators does not matter as long as only
unary operators are involved, because there is only one way to parse
them--innermost first. Thus, $++i
means $(++i)
and
++$x
means ++($x)
. However, when another operator follows
the operand, then the precedence of the unary operators can matter.
Thus, $x**2
means ($x)**2
, but -x**2
means
-(x**2)
, because `-' has lower precedence than `**'
while `$' has higher precedence.
Here is a table of the operators of awk
, in order of increasing
precedence:
in
.
The relational operators are `<', `<=', `==', `!=', `>=' and `>'.
The I/O redirection operators are `<', `>', `>>' and `|'.
Note that I/O redirection operators in print
and printf
statements belong to the statement level, not to expressions. The
redirection does not produce an expression which could be the operand of
another operator. As a result, it does not make sense to use a
redirection operator near another operator of lower precedence, without
parentheses. Such combinations, for example `print foo > a ? b :
c', result in syntax errors.
Go to the previous, next section.