The character-encoding tag has no usage guidance.
0
votes
0answers
9 views
Character encoding keyboard suddenly changes
Now and then I lose the Swedish encoding on my Linux platform Kubuntu / just suddenly the character set on the keyboard is changed. For instance when I press the key for the Swedish letter ä, got it ...
0
votes
0answers
12 views
Why is LXDE displaying only boxes for font - and how to fix
A picture is worth 1000 words:
This is fedora 20 running LXDE using X2GOServer, with X2GO client on windows. I am on a VPS at ramnode.
I assume it's an issue with encoding, but I just can't ...
2
votes
1answer
10 views
What encoding am I using? $LANG doesn't have an encoding
It seems like typically:
echo $LANG
results in something like this:
en_US.UTF-8
What encoding is used when the result does not specify an encoding?
echo $LANG
en_US
How do I figure out ...
2
votes
0answers
29 views
some cyrillic file names got broken after moving from server to server
I moved my website from one server to another and encounter a problem with cyrillic filenames. Some cyrillic URLs in wordpress are fine and some are now broken. It used to be OK on previous server.
...
0
votes
0answers
14 views
iconv adding redundant Shift-in 0x0F
we are using the below command to convert the codepage from utf8 to cp935
iconv filename -f utf8 -t Cp935 > out.filename
the end file will be in Cp935 encoding for Chinese. At times this ...
2
votes
1answer
41 views
UTF-8 not working over SSH
I have configured my terminal on OSX to show emoji properly. When I open up a terminal I can type and see emoji properly. My locale settings for OSX are shown below. When I start a tmux session it ...
1
vote
1answer
42 views
Unix command to convert character encoding in a .csv file
I need a Unix command to convert a .csv file that is in UNICODE format to ANSI format.
The file is imported from the Cognos environment and I am unable to make any changes with the format in cognos ...
2
votes
1answer
31 views
Running grep for a string with non-rendering characters
So I have a log.txt files which I can search in Notepad++ on Windows, however due to the sheer number of them, I'd like to be able to run grep on them. However, the data is formatted with ...
4
votes
3answers
385 views
“mv” file with garbled name by inode number?
I have several files with encoding issues in their file names (German umlauts, burned on CD with Windows, read by Windows and synced to Linux with Seafile. Something, somewhere went wrong...).
Bash ...
3
votes
1answer
44 views
How to use grep with files in arbitrary encoding, without locale installed and without sudo?
I have utf-8 locale on my computer, and i want to search in KOI8-R encoded files. But i don't have KOI8-R locale installed. And i don't have sudo rights to install this locale.
Is it possible to ...
0
votes
0answers
27 views
Polish–Vietnamese mojibake in a .txt file, using LibreOffice, mousepad, and vim
I have a trouble with some .txt file. When I try open it with mousepad, vim or LibreOffice Writer in every case some Polish letters are changing to another, incorrectly (i.e. correct char ś and ć are ...
5
votes
1answer
121 views
Untar filenames in a character encoding different from encoding used in the filesystem
I occasionally get tarballs where the filenames are encoded in ISO-8859-1 or some other pre-Unicode scheme. My system uses UTF-8, so when I untar these archives with the usual options (tar xvf ...
2
votes
1answer
69 views
Encoding problem on aix system when logging using log4j2
I am using log4j2 to log message on aix system. In log when I use command less I can see:
...|LU?N
...|NOV�K
...|ZEM�NEK
when I copy same file on my ubunty system and use same program I can see:
...
3
votes
0answers
48 views
Get consistent encoding for all files in directory
I have a directory containing lots of csv files from various vendors with two different encodings:
ASCII Text / UTF-8
UCS2 / UTF-16 little endian
I'd like to use grep, awk, sed and other utilities ...
6
votes
3answers
718 views
Convert an ASCII file with octal escapes for UTF-8 codes to UTF-8
I have a text file which is an ASCII file itself, but contains octal escape sequences representing codes in utf-8:
\350\207\252\345\212\250\346\216
Is there some program or command that can ...
0
votes
0answers
138 views
Nothing helps Gnome Terminal to not default to ANSI_x3.4-1968
I would really like gnome-terminal to default to a UTF-8 encoding, but whatever I do, I can't get it to do it.
I don't have sudo access, so I cannot edit /etc/environment. I'm on a university-managed ...
2
votes
3answers
219 views
Why are file names surrounded with “â” sign?
After adding alias rm='rm -i' to my ~/.bashrc file (because, when I removed a file, it wasn't asking for confirmation), file names are surrounded with "â" signs as in the example below:
rm: cannot ...
-1
votes
1answer
50 views
Usefulness of --from-code in command iconv
Is the --from-code option of the iconv command useful (http://man7.org/linux/man-pages/man3/iconv.3.html)?
I mean, I don't see any difference if I specify this parameter (--from-code 'iso-8859-1' or ...
4
votes
1answer
72 views
Gedit displays text, but cat tells a different story
I am trying to sort a dictionary from Grady Ward's Moby and have encountered a bit of a problem. When I open it in gedit, I get:
abaca×N
abaciscus×N
abacist×N
aback×v
abacli×p
Abaco×N
abactinal×A
...
1
vote
0answers
22 views
Understanding script and font relation (character-map)
When I use charactermap, I found that if I change font, only Latin script changes. Character by all other scripts sounds independent with change in font!
So, I want to know what is relationship ...
1
vote
1answer
54 views
How to format password using SCP
I'm currently writing a Script to upload files to my server via SCP.
I have the following line:
sshpass -p password!% scp -P 1234 ./test.text user@mydyndns:~/test.txt
But everytime I try that I ...
0
votes
0answers
26 views
Searching text data in Hebrew, Arabic or Aramaic
How can I search for a (sub-)string in a database that has several languages, including Hebrew?
I am assuming that the text being right-to-left is a question of the presentation, but the logical ...
1
vote
1answer
70 views
Can not get the file names with convmv
I have just uploaded some files on a FreeBSD server. Some files have the ??? into their names. The files came from a windows machine, so I try this:
$ convmv -f cp850 -t utf-8 *
Starting a dry run ...
2
votes
1answer
39 views
Cyrillic file names broken after upgrading Slackware
After upgrading glibc and some other libraries on Slackware instead of cyrillic names I see the garbage:
| Was before | Is now |
|------------+------------------|
| Вождение | ...
2
votes
1answer
158 views
Convert TeX accents to UTF-8 in Python
edit below
First of all, I am quite sure this is a duplicated question. But I have no idea how to search for it. I spent quite a long time already.
I am reading with python from a .txt file the path ...
0
votes
1answer
195 views
LMDE2 / MATE locale charset inconsistency
I have a machine with Linux Mint Debian Edition 2 "Betsy" (installed as RC, with all available updates applied) and MATE desktop environment. My problem is that in some applications special characters ...
0
votes
1answer
70 views
make `tar` archive from NTFS with non-ascii file names
Loaded a live disk into a Windows based machine with NTFS files. Being a Linux user for some years, using tar came to my mind instantly and made some backup archives from partitions and stored them ...
4
votes
4answers
364 views
Prevent tail from modifying the charset of the terminal
In a Linux terminal (CentOS) I am using the command tail --follow=name my-rolling-file.log in order to see the logs of my application.
Sometimes in the log there is some binary data dumped (I dump ...
0
votes
1answer
42 views
Error not found [closed]
I wrote this script to encode / encrypt data from a sql query by changing the names of tables and fields (I played on the ASCII character codes).
#! /bin/bash
chr() {
[ "$1" -lt 256 ] || return 1
...
28
votes
2answers
903 views
find(1): how is the star wildcard implemented for it to fail on some filenames?
In a file system where filenames are in UTF-8, I have a file with a faulty name; it is displayed as: D�sinstaller, actual name according to zsh: D$'\351'sinstaller, Latin1 for Désinstaller, itself a ...
0
votes
1answer
132 views
iconv can't convert circled digits to/from Japanese encodings
The unicode range of circled digits (U+2460 .. U+2468) cannot be converted to, or from, any of the Japanese encodings (EUC-JP, Shift-JIS, ISO-2022-JP), even though they exist there, and I run across ...
0
votes
2answers
539 views
Check whether text contains non-ASCII characters in a shell script
I have a file, I want to determine if it contains only English ASCII characters.
I thought this is a very common question, but when I googled it, there is no direct answer or related.
According to ...
1
vote
1answer
183 views
cp: cannot stat error - when filename has Asian characters
I am simply trying to copy files using cp -r /home/user/source/ /home/user/destination/ but it throws me the cp: cannot stat /source/filename.xxx error for some of the files. When I searched for this ...
1
vote
4answers
434 views
How to recognize which ASCII character in hex is this?
We have a textfile that we want to clear from "bad" characters. If we open it with vim (with ":set number"):
57000044 zo¥<9a>¥ge¥o¥graph¥i¥cal¥ly
39999999 pariá¹<83>Å<9b>a
The ...
1
vote
2answers
69 views
How to only leave in given characters in a file?
Purpose:
tr -cd '\12\40-\176'
but with accents, ex.: á, í, ű, ő, ö, ü, ű, ú, ó, é.
In octal:
"á" is 341
"\n" is 12
So I tried:
[user@notebook ~]$ printf 'XXXá\nXXX' | tr -cd '\12\341' | ...
1
vote
1answer
70 views
Any way to interpret pstree?
-bash-3.2$ pstree 29838
RBcron.shâââjavaâââ15*[{java}]
-bash-3.2$ pstree 24423
bashââ¬âRBcron.shâââjavaâââ15*[{java}]
ââvim
-bash-3.2$
Any way to interpret this? Why would this happen?
2
votes
1answer
245 views
Question marks instead of national letters in console
I have Netgear Stora MS2000 (which uses a modified version of Red Hat) and have strange font issues while connecting by SSH. Note that this occurs both when connecting by PuTTY from Windows and by ...
1
vote
1answer
95 views
How do I edit text in Cyrillic / Russian on OpenBSD?
I'm trying to make it possible to easily edit text in Cyrillic / Russian on a remote OpenBSD 5.2 amd64.
I'm using emacs-21.4p18-no_x11, which doesn't show any Cyrillic characters whatsoever when I ...
3
votes
3answers
1k views
How to convert unknown-8bit file to utf8
I have a .srt file that displays as gibberish when I open it in gEdit in ubuntu.
So I want to convert it to utf8 to be able to read it.
When I try to figure out what the encoding it give:
file -i ...
1
vote
1answer
154 views
Rename files with non-utf8 characters using detox
I am trying to use detox in the following way
detox -s utf_8-only -r -v -n
to detect invalid filenames on my system. The command performs mostly ok. Unfortunately it falsely detects also German ...
4
votes
2answers
308 views
Characters visible in vi, but not in cat.
I have a 300 lines file with ^@ character between each characters in the file.
(I cannot post the entire contents for security reasons, so I am pasting only the first line)
[mercury@app01 ftp_logs]$ ...
21
votes
5answers
3k views
Generating a random password; why isn't this portable?
I want to generate a random password, and am doing it like so:
</dev/urandom tr -dc [:print:] | head -c 64
On my laptop, which runs Ubuntu, this produces only printable characters, as intended. ...
1
vote
1answer
228 views
Special characters in a web feed
I am running a script to check my gmail from the command line and display the first 35 characters in the title.
curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | \
grep ...
1
vote
1answer
244 views
script to parse integer and convert it to char code
I am noob on Unix. I've a very unique problem where and really appreciate your help. I need to parse string containing hash code value and convert the hash code is equivalent char representation. Here ...
0
votes
2answers
98 views
Find out which encoding method less uses to display the content of a text file?
There is a plain text file, which Emacs can't display correctly, but less can.
I wonder if less automatically detects the encoding method for each individual text file, or just use a single default ...
3
votes
1answer
410 views
How do I view cp1251 text file in UTF-8 console?
Attempt 1:
$ less subs.srt
"subs.srt" may be a binary file. See it anyway?
<C8><F2><E0><EB><FC><FF><ED> ...
Attempt 2:
$ LANG=ru_RU.CP1251 less subs.srt
...
1
vote
0answers
91 views
How do I configure medit to display UTF-8?
My environment is set to use UTF-8. I produce a file in UTF-8 encoding. It works correctly on the terminal.
For example:
$ echo ééé > test
$ cat test
$ echo é | od -t x1
0000000 c3 a9 0a
...
1
vote
1answer
270 views
iconv terminated by signal 7
I was running a little bash script on some text files:
find . -name "*.dat" -exec iconv --from-code='UTF-8' --to-code='ASCII//TRANSLIT' {} --output={} \;
My machine is a Ubuntu 14.04 LTS.
After a ...
2
votes
0answers
127 views
iconv cannot replace Ø
It appears that iconv cannot, for example, replace the letter Ø. It was also noted in the second answer to http://stackoverflow.com/questions/3371697/replacing-accented-characters-php
I have two ...
3
votes
2answers
120 views
Why can't I grep this command line option?
The output of man icc contains
-Ldir
Tells the linker to search for libraries...
Doing man icc | grep "\-L" gives back the "-Ldir" line while man icc | grep "\-Ldir" does not. What am ...
