Skip to main content

New Lines - Windows/Unix/Linux/MacOS - viM

If you deal with scripts and other text files and move between platforms you probably discovered this "issue".
Only the founding developers can explain why they chose what they did - googling about will show you a couple of different explanations - whatever the reasons, here are the differences and how to convert.

The formats

The Characters in use (referenced in OS info below)

  • LF
    • zoeyburger
    • Usually referred to as LF  or Line feed
    • Ascii code decimal 10
    • Hex: A or 0xA
    • Octal: 12 or O12
    • Typical Escaped character in many shells and languages: \n
  • CR
    • Usually referred to as CR or Carriage Return
    • Ascii code decimal 13
    • Hex D or 0xD
    • Typical Escaped character in many shells and languages: \r

Unix, Linux, and Modern MacOS - The POSIX standard

Each Line ends with a single character:  LF
Most programming languages will understand/interpret this format properly.
Simple Windows programs, like the built in Notepad will not show this properly.

Windows (and DOS)

Each line ends with two consecutive characters in this order: CRLF

Classic (Ancient) MacOS

Each Line ends with a single character:  CR
This is no longer much of a practical issue, it is from the dark ages of Mac.

Windows - Use Advanced (Free) Editors

If you primarily work on Windows, then all you need is a good editor of sorts, like Notepad++ or Visual Studio Code - they can handle conversion and detection for you. If you are in a hurry and just annoyed that a file opens in Notepad and looks like one long line, just open the file in Wordpad instead, that shoul let you read it. (But really, install Notepad++)

Vim (vi Improved)

vi is the "native" CLI/Shell based editor for many Unix and Linux users, it has been around for a long time, vim is the modern and quite capable version - and it is my go-to on any non-windows system, any CLI for sure, now even exists for Windows with a graphical UI (gvim).
It is not an easy editor to learn in a hurry, you need to take some time reading, testing, and understanding how it is fundamentally different (than say pico or nano), vi is not one to figure out by clicking around. That said, once you grasp what it does, it has great powers that will to the untrained eye seem magical.

If you open a file on a system and you see a bunch of ^M at each line ending, here is how you can replace them all in vi with nothing, so that you end up with LF only endings:

  • start a command with colon :
  • % for all lines
  • s/ for substitute
  • Ctrl+v, then hit Enter - this will produce a ^M
  • then // to substitute with nothing
  • so your command will look like:  % s/^M//

Shell / CLI

There are a ton of ways to do these conversions - here are a couple options

  • To create LF only from CRLF (Simply delete any and all CR)
    • cat windowsfile | sed 's/\r//' > nixfile
    • cat windowsfile | tr -d '\015' > nixfile
    • cat windowsfile | dos2unix > nixfile

HTTP headers and MIME Formatting

This has nothing to do with files and scripts editors - but just though i would point out that the standards used in MIME (email) formatting and HTTP headers utilizes CRLR (\r\n), it is a common mistake for unseasoned script writers and such to forget this..  Make sure to read up on the rules and RFCs if you are creating mail senders or http services.

Labels:

Comments

Post a Comment

Popular posts from this blog

Removing Domain - Office 365 / Azure AD Tenant

Recently I had an interesting experience and challenge, removing a domain from an Azure AD (Office 365) Tenant which had been around for years, switching all the users to another domain for logins/UPN. A normal procedure for this should be simple: Change UPN for all users and groups Change any associated apps, email, and other resources Remove Domain (This can be done from Azure Portal, or from Office 365 Admin). The issue for me was that there was resources associated with some users, which I could not find what recourses or how to clear it up. In Azure Portal, Azure AD, Custom Domains - it would not let me delete at all, just showed me a link to the list of users in violation. In office 365 Admin, Settings, Domains - I was able to initiate a Delete action, once, with a supposed automatic removal action. After several hours this failed, and it now remained in a failed state that did not let me try again from UI. So I started digging with PowerShell - I found it most usable with the MS
Post a Comment
Read more

Cisco UCS Mini - Add Extender Chassis

If you happen to own a UCS Mini Setup, a 5108 Chassis with two Fi 6324 or similar, and you are looking for documentation on how to add another 5108 Chassis with fabric extenders (2204XP in my case), then Cisco really does not have much out there, nor is there a lot of googlable information either (Everything you find is related to standalone Fabric Interconnects and "standard" UCS). Even after calling TAC, it took a while to get something, and what they told us was not even accurate. So here is how we did it, and it worked, came up without any interruption to current chassis, network, or running profiles. Equipment Of course we used our Cisco vendor to spec the equipment, but just for reference here is the list of what we had and what we added: Original Setup 5108 Chassis  Fi 6324 (Qty 2) Ports 1-2 for Fibre Channel, and 3-4 for Ethernet (MMF) Connected to a stack of switches and pair of FC switches/SAN Running UCS version 4.0.1 (Fairly recently upgraded as of M
Post a Comment
Read more

Linux/Unix - Create a local Certificate Authority (CA)

I get these questions all the time - people know i have some runtime with certificates and such - one question is "Can't i just issue my own certs?" - and the answer of course is yes - but I always make sure to add that it won't be any use on a public web site since no-one will trust it. So setting up your own CA is not "generally useful", it is more if you need some specific things, like issuing certificates with a single signing source for client logins or similar. Most business will have a  couple of Windows Domain controllers, if you need to sign certs for a limited set of users, what you should do is make sure some system in your windows domain runs Certificate Services, then issue certs from there, make sure any non-domain-members has a trust for that CA. If you actually do need to set up you own CA, here is one way to do it Procedure to set up your own local CA The common name for the CA cert must NOT be the same as a domain name or anything e
Post a Comment
Read more