Skip to main content

New Lines - Windows/Unix/Linux/MacOS - viM

If you deal with scripts and other text files and move between platforms you probably discovered this "issue".
Only the founding developers can explain why they chose what they did - googling about will show you a couple of different explanations - whatever the reasons, here are the differences and how to convert.

The formats

The Characters in use (referenced in OS info below)

  • LF
    • zoeyburger
    • Usually referred to as LF  or Line feed
    • Ascii code decimal 10
    • Hex: A or 0xA
    • Octal: 12 or O12
    • Typical Escaped character in many shells and languages: \n
  • CR
    • Usually referred to as CR or Carriage Return
    • Ascii code decimal 13
    • Hex D or 0xD
    • Typical Escaped character in many shells and languages: \r

Unix, Linux, and Modern MacOS - The POSIX standard

Each Line ends with a single character:  LF
Most programming languages will understand/interpret this format properly.
Simple Windows programs, like the built in Notepad will not show this properly.

Windows (and DOS)

Each line ends with two consecutive characters in this order: CRLF

Classic (Ancient) MacOS

Each Line ends with a single character:  CR
This is no longer much of a practical issue, it is from the dark ages of Mac.

Windows - Use Advanced (Free) Editors

If you primarily work on Windows, then all you need is a good editor of sorts, like Notepad++ or Visual Studio Code - they can handle conversion and detection for you. If you are in a hurry and just annoyed that a file opens in Notepad and looks like one long line, just open the file in Wordpad instead, that shoul let you read it. (But really, install Notepad++)

Vim (vi Improved)

vi is the "native" CLI/Shell based editor for many Unix and Linux users, it has been around for a long time, vim is the modern and quite capable version - and it is my go-to on any non-windows system, any CLI for sure, now even exists for Windows with a graphical UI (gvim).
It is not an easy editor to learn in a hurry, you need to take some time reading, testing, and understanding how it is fundamentally different (than say pico or nano), vi is not one to figure out by clicking around. That said, once you grasp what it does, it has great powers that will to the untrained eye seem magical.

If you open a file on a system and you see a bunch of ^M at each line ending, here is how you can replace them all in vi with nothing, so that you end up with LF only endings:

  • start a command with colon :
  • % for all lines
  • s/ for substitute
  • Ctrl+v, then hit Enter - this will produce a ^M
  • then // to substitute with nothing
  • so your command will look like:  % s/^M//

Shell / CLI

There are a ton of ways to do these conversions - here are a couple options

  • To create LF only from CRLF (Simply delete any and all CR)
    • cat windowsfile | sed 's/\r//' > nixfile
    • cat windowsfile | tr -d '\015' > nixfile
    • cat windowsfile | dos2unix > nixfile

HTTP headers and MIME Formatting

This has nothing to do with files and scripts editors - but just though i would point out that the standards used in MIME (email) formatting and HTTP headers utilizes CRLR (\r\n), it is a common mistake for unseasoned script writers and such to forget this..  Make sure to read up on the rules and RFCs if you are creating mail senders or http services.

Labels:

Comments

Post a Comment

Popular posts from this blog

Active Directory Account Lockout - Narrowing Down the source

If you are in a all-windows shop where everything is nice and neat, everybody has a proper domain membership and all authentication is SSO or Windows Integrated, then you probably do not have much of a problem with repeated account lockouts. On the other hand, if you are in a mixed environment, lots of :Linux, Mac, and unmanaged Wintendo, then you probably run into some users that manage to Lock themselves out frequently - typically for several days in a row after the account password had been changed. Reasons can be plenty fold - typically saved credentials somewhere, like a git client, sql-server client, email client, rdp-manager, smbfs-automount, or anything that tries a bunch of logins when you start it up, or keeps trying in the background. As a sysadmin, you don't have time to narrow it down for the end user - but they will be adamant it is not their fault, so you probably need to prove that "Yes it is" - so I use powershell to grab 4740 events from Domain Con...
Post a Comment
Read more

Campground Networking

I've travelled a couple years full time in an RV, working remotely. This can be a challenger, many campgrounds have poor Wi-Fi setups, and cell service is not always great (Do not plan on doing any work from the Grand Canyon National Park). Calling ahead and asking usually does not reveal accurate information, you are best off using campground reviews, search them for WIFI and read what people say.  The best network I have seen was at Eagles Landing RV Park in Holt, FL (Pan handle) http://eagleslandingrvpark.com/. Still not perfect. At poor sites, more than once did I offer my assistance in trying to configure and improve, but even the places which have no vendor maintain their system do not want any other hands on it. A couple of times I helped out anyway, default passwords on routers, so I upgraded their firmware, disabled 802.11b, and set a password so no-one else would mess with it. My RV network setup is not of a common type, you sort of have to be a bit of a network-guy to us...
1 comment
Read more

Cisco UCS Mini - Add Extender Chassis

If you happen to own a UCS Mini Setup, a 5108 Chassis with two Fi 6324 or similar, and you are looking for documentation on how to add another 5108 Chassis with fabric extenders (2204XP in my case), then Cisco really does not have much out there, nor is there a lot of googlable information either (Everything you find is related to standalone Fabric Interconnects and "standard" UCS). Even after calling TAC, it took a while to get something, and what they told us was not even accurate. So here is how we did it, and it worked, came up without any interruption to current chassis, network, or running profiles. Equipment Of course we used our Cisco vendor to spec the equipment, but just for reference here is the list of what we had and what we added: Original Setup 5108 Chassis  Fi 6324 (Qty 2) Ports 1-2 for Fibre Channel, and 3-4 for Ethernet (MMF) Connected to a stack of switches and pair of FC switches/SAN Running UCS version 4.0.1 (Fairly recently upgraded as of M...
Post a Comment
Read more