Note that, unescaped, this expands on the client side.

Problematic code:

ssh host "echo $HOSTNAME"

Correct code:

ssh host "echo \$HOSTNAME"

or

ssh host 'echo $HOSTNAME'

Rationale:

Bash expands all arguments that are not escaped/singlequoted. This means that the problematic code is identical to

ssh host "echo clienthostname"

and will print out the client's hostname, not the server's hostname.

By escaping the $ in $HOSTNAME, it will be transmitted literally and evaluated on the server instead.

Exceptions

If you do want your string expanded on the client side, you can safely ignore this message.

Keep in mind that the expanded string will be evaluated again on the server side, so for arbitrary variables and command output, you may need to add a layer of escaping with e.g. printf %q.