Bashism
See this post for an overview.
They are history expansion characters. See this article for a quick introduction. Full reference is provided in this section of the bash manual.