current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
Join the Stack Overflow Community
Stack Overflow is a community of 6.6 million programmers, just like you, helping each other.
Join them; it only takes a minute:
Sign up
up vote 12 down vote favorite
3

As we use # for inserting comments in Python, then how does Python takes:

# -*- coding: utf-8 -*-

differently?

share|improve this question
    
this actually happens often with software. They actually do some elementary parsing of comments and look for specific commands. Another example i have in mind is Hypermesh but i am sure there are many more. – Ev. Kounis 8 hours ago
    
You can think of it as a preprocessor that runs before the parser that peeks at the file and decides how it should be decoded. Then the parser itself starts and skips the line because its a comment. Some unixy text editors do the same thing to know how the editor should open the file. – tdelaney 8 hours ago
1  
@Ev.Kounis Perhaps the most prominent example: en.wikipedia.org/wiki/Shebang_(Unix) – deceze♦ 8 hours ago
5  
It's worth noting that the -*- parts are completely optional, as far as Python is concerned, but including them seems to be customary. The docs say it "is recognized also by GNU Emacs", which suggests that that's where it comes from (an example of what @tdelaney was saying about text editors), but I've seen it (and used it myself) in code that was never touched by Emacs. – Tim Pederick 8 hours ago
    
@TimPederick: I've used the VIM variant of the comment often enough. Are you certain there wasn't an emacs user on the team somewhere that appreciated having their editor auto-configured when editing the file? – Martijn Pieters♦ 7 hours ago
up vote 18 down vote

Yes, it is also a comment. And the contents of that comment carry special meaning if located at the top of the file, in the first two lines.

From the Encoding declarations documentation:

If a comment in the first or second line of the Python script matches the regular expression coding[=:]\s*([-\w.]+), this comment is processed as an encoding declaration; the first group of this expression names the encoding of the source code file. The encoding declaration must appear on a line of its own. If it is the second line, the first line must also be a comment-only line.

Note that it doesn't matter what codec should be used to read the file, as far as comments are concerned. Python would normally ignore everything after the # token, and in all accepted source code codecs the #, encoding declaration and line separator characters are encoded exactly the same as they are all supersets of ASCII. So all the parser has to do is read one line, scan for the special text in the comment, read another if needed, scan for the comment, then configure the parser to read data according to the given codec.

Given that the comment is required to be either the first or second in the file (and if it is the second line, the first line must be a comment too), this is entirely safe, as the configured codec can only make a difference to non-comment lines anyway.

share|improve this answer
    
So the real question becomes: why do we use # -*- coding: X -*- instead of # coding: X? – J. C. Leitão 7 hours ago
2  
@J.C.Leitão: you don't have to. Anything that matches the regular expression would work. But if you are using Emacs as your editor, then that comment also informs that editor what codec to use. – Martijn Pieters♦ 7 hours ago
    
` what codec the file is saved as` shouldn't be what encoding the file is saved as? – Nizam Mohamed 7 hours ago
2  
@ShravilPotdar: There's loads. There is the shebang line that Unix systems use, and the Windows py launcher will look at the same info. As mentioned, many editors can be configured using text in comments (not just what codec to use, but many other aspects as well, see the emacs and vim docs). There are probably more. – Martijn Pieters♦ 6 hours ago
1  
@R.M.: no, multi-byte codecs are not supported, for this very reason. From PEP 263: Any encoding which allows processing the first two lines in the way indicated above is allowed as source code encoding, this includes ASCII compatible encodings as well as certain multi-byte encodings such as Shift_JIS. It does not include encodings which use two or more bytes for all characters like e.g. UTF-16. The reason for this is to keep the encoding detection algorithm in the tokenizer simple. – Martijn Pieters♦ 3 hours ago
up vote 12 down vote

See encoding declarations in the Python Reference Manual:

If a comment in the first or second line of the Python script matches the regular expression coding[=:]\s*([-\w.]+), this comment is processed as an encoding declaration; the first group of this expression names the encoding of the source code file.

(Emphasis mine)

So yes, it is a comment, a special one. It is special in that the parser will try and act on it and not ignore it as it does for comments not in the first or second line. Take, for example, an unregistered encoding declaration in a sample file decl.py:

# # -*- coding: unknown-encoding -*-
print("foo")

If you try and run this, Python will try and process it, fail and complain:

python decl.py 
  File "decl.py", line 1
SyntaxError: encoding problem: unknown-encoding
share|improve this answer
1  
But if you were to register unkown-encoding as an encoding, say, with a .pth file, then that codec is actually loaded and used. This provides a very nice and interesting opportunity for pre-parse code processing. – Martijn Pieters♦ 8 hours ago
    
Indeed @MartijnPieters I mainly added that as a code example that Python processes the declaration, not to make any other claims for it. – Jim Fasarakis-Hilliard 8 hours ago
1  
github.com/dropbox/pyxl would be an example of what @MartijnPieters is referring to. – Łukasz Rogalski 8 hours ago

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.