Skip to main content

New Lines - Windows/Unix/Linux/MacOS - viM

If you deal with scripts and other text files and move between platforms you probably discovered this "issue".
Only the founding developers can explain why they chose what they did - googling about will show you a couple of different explanations - whatever the reasons, here are the differences and how to convert.

The formats

The Characters in use (referenced in OS info below)

  • LF
  • zoeyburger
    • Usually referred to as LF  or Line feed
    • Ascii code decimal 10
    • Hex: A or 0xA
    • Octal: 12 or O12
    • Typical Escaped character in many shells and languages: \n
  • CR
    • Usually referred to as CR or Carriage Return
    • Ascii code decimal 13
    • Hex D or 0xD
    • Typical Escaped character in many shells and languages: \r

Unix, Linux, and Modern MacOS - The POSIX standard

Each Line ends with a single character:  LF
Most programming languages will understand/interpret this format properly.
Simple Windows programs, like the built in Notepad will not show this properly.

Windows (and DOS)

Each line ends with two consecutive characters in this order: CRLF

Classic (Ancient) MacOS

Each Line ends with a single character:  CR
This is no longer much of a practical issue, it is from the dark ages of Mac.

Windows - Use Advanced (Free) Editors

If you primarily work on Windows, then all you need is a good editor of sorts, like Notepad++ or Visual Studio Code - they can handle conversion and detection for you. If you are in a hurry and just annoyed that a file opens in Notepad and looks like one long line, just open the file in Wordpad instead, that shoul let you read it. (But really, install Notepad++)

Vim (vi Improved)

vi is the "native" CLI/Shell based editor for many Unix and Linux users, it has been around for a long time, vim is the modern and quite capable version - and it is my go-to on any non-windows system, any CLI for sure, now even exists for Windows with a graphical UI (gvim).
It is not an easy editor to learn in a hurry, you need to take some time reading, testing, and understanding how it is fundamentally different (than say pico or nano), vi is not one to figure out by clicking around. That said, once you grasp what it does, it has great powers that will to the untrained eye seem magical.

If you open a file on a system and you see a bunch of ^M at each line ending, here is how you can replace them all in vi with nothing, so that you end up with LF only endings:

  • start a command with colon :
  • % for all lines
  • s/ for substitute
  • Ctrl+v, then hit Enter - this will produce a ^M
  • then // to substitute with nothing
  • so your command will look like:  % s/^M//

Shell / CLI

There are a ton of ways to do these conversions - here are a couple options

  • To create LF only from CRLF (Simply delete any and all CR)
    • cat windowsfile | sed 's/\r//' > nixfile
    • cat windowsfile | tr -d '\015' > nixfile
    • cat windowsfile | dos2unix > nixfile

HTTP headers and MIME Formatting

This has nothing to do with files and scripts editors - but just though i would point out that the standards used in MIME (email) formatting and HTTP headers utilizes CRLR (\r\n), it is a common mistake for unseasoned script writers and such to forget this..  Make sure to read up on the rules and RFCs if you are creating mail senders or http services.

Comments

Popular posts from this blog

Cisco UCS Mini - Add Extender Chassis

If you happen to own a UCS Mini Setup, a 5108 Chassis with two Fi 6324 or similar, and you are looking for documentation on how to add another 5108 Chassis with fabric extenders (2204XP in my case), then Cisco really does not have much out there, nor is there a lot of googlable information either (Everything you find is related to standalone Fabric Interconnects and "standard" UCS). Even after calling TAC, it took a while to get something, and what they told us was not even accurate. So here is how we did it, and it worked, came up without any interruption to current chassis, network, or running profiles.
Equipment Of course we used our Cisco vendor to spec the equipment, but just for reference here is the list of what we had and what we added:

Original Setup 5108 Chassis Fi 6324 (Qty 2)Ports 1-2 for Fibre Channel, and 3-4 for Ethernet (MMF)Connected to a stack of switches and pair of FC switches/SANRunning UCS version 4.0.1 (Fairly recently upgraded as of May 2019)Added Equip…

Linux/Unix - Create a local Certificate Authority (CA)

I get these questions all the time - people know i have some runtime with certificates and such - one question is "Can't i just issue my own certs?" - and the answer of course is yes - but I always make sure to add that it won't be any use on a public web site since no-one will trust it.
So setting up your own CA is not "generally useful", it is more if you need some specific things, like issuing certificates with a single signing source for client logins or similar.

Most business will have a  couple of Windows Domain controllers, if you need to sign certs for a limited set of users, what you should do is make sure some system in your windows domain runs Certificate Services, then issue certs from there, make sure any non-domain-members has a trust for that CA.

If you actually do need to set up you own CA, here is one way to do it
Procedure to set up your own local CA The common name for the CA cert must NOT be the same as a domain name or anything else you…

Home made SAN Migration

The topic sounds more elaborate than it is - alternative title could be "Hackjob SAN volume backup and restore".
The SetupIn a legacy-style SAN and Compute setup, I have an EMC Unity 450F box deployed with Fibre Channel (FC) to a Cisco UCS (Unified Compute System). I am booting the UCS blades off the SAN, running vmware with Block/LUN DataStores, and one blade running Windows.
I also have a Dell R740 server in the mix, with a Qlogic HBA as well as onboard storage.
The Situation Not in production yet, but we had spun up a few VMs, and all our blades had been installed, esx and vcenter running, a few VM's, and including the physical Windows blade, and the R740.

Then we discovered that the Unity had no SFP+ ports in it, and I need to do replication - swearing my vendor up and down, I call EMC, and they are sending out NIC modules and guy to install it. BUT, because we have to remove a module to insert a new one, the whole SAN box has to be reset to Factory setting (!!!). My…