Wednesday, December 21, 2022

Where to find .gitattributes on Windows 10/11 using Git, IntelliJ 2022 and WSL Ubuntu to fix CRLF (\r\n) command not found: convert to LF line endings on checkout

Introduction

Problem: when checking out a project in IntelliJ in Windows, all files are checked out with Window's newline CRLF (\r\n).

But if you then open a terminal in IntelliJ which runs WSL (Ubuntu) and you want to run a bash script like this shell script, you'll get this error:

#!/bin/sh

set -e

[unrelated stuff deleted]

It will fail with: 

./deploy.sh: line 2: $'\r': command not found

: invalid optione 3: set: -

set: usage: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]

Those errors are caused by the shell script having CRLF instead of just LF, which Ubuntu/Mac OS expects.

Sidenote: I made softlink from /bin/sh to bash in my system because dash does not support several features, see here


So I tried setting IntelliJ's Code Style to use \n for newlines and line endings and line separator, also for new projects like mentioned here.

But still after creating a new project from Git, all files including the above script was set to CRLF. You can see this at the bottom of the screen in this screenshot:


I also realised that I don't want all files to have just LF, because I noticed doing that manually, on my commit of those files, I had to commit these too, as the newline changed. But I didn't want to bother other teammembers on Mac Books with this unnecessary change. Similar to what happens when you do a dos2unix on the file.
So the next to try was to get Git on my machine to handle it correctly. And thus hopefully IntelliJ too.
The .gitattributes file seemed a very good candidate: a suggestion was to change the git config 'core.autocrlf'. But that meant all the local files were getting changed still if I understood correctly, which I don't want.

Solution

It cost a lot of effort to find out where the gitattributes file is currently; and I wanted to change it for all my Git commands in the future, not just for one project.
What I wanted to change it to is mentioned here:

# Convert to LF line endings on checkout.
*.sh text eol=lf
**/*.sh eol=lf

Those lines specify to change all end of lines (newlines) of files ending with .sh to be checked out with a LF (\n).
I also added the 3rd line myself, to specify subdirectories, but maybe that was not needed.

It was hard to find the correct location for the gitattributes (or .gitattributes file which made it all more confusing), this official Git text wasn't super-clear on it: 

Finally I found my gitattributes file on Windows here:

C:\git\etc\gitattributes

C:\git is actually where I installed the Git client. And it is the one IntelliJ uses (but not the one the terminal shell in IntelliJ uses!) 

And there were already quite a few settings in there like:

*.RTF diff=astextplain
*.doc diff=astextplain
*.DOC diff=astextplain

I just appended these:

**/*.sh eol=lf
*.sh eol=lf

Restarted IntelliJ to be sure. And yes, after a complete full new checkout of my Git project, Java and Kotlin files were still having CRLF line endings, and my shell script had the LF ending.

Sunday, December 18, 2022

Docker build with Git command running in CircleCI failing with: Fatal: No names found, cannot describe anything, invalid argument, for "-t, --tag" flag: invalid reference format

Introduction

Context: Docker, CircleCI, Github.

The Docker build command 

docker build -f .circleci/Dockerfile -t $AWS_ACCOUNT_ID.ecr.$AWS_DEFAULT_REGION.amazonaws.com/${CIRCLE_PROJECT_REPONAME}:`git describe --tags` -t $AWS_ACCOUNT_ID.ecr.$AWS_DEFAULT_REGION.amazonaws.com/${CIRCLE_PROJECT_REPONAME}:${CIRCLE_BUILD_NUM} -t $AWS_ACCOUNT_ID.ecr.$AWS_DEFAULT_REGION.amazonaws.com/${CIRCLE_PROJECT_REPONAME}:${CIRCLE_SHA1} .

was failing with this message:

fatal: No names found, cannot describe anything.

invalid argument "********************************************/my-project:" for "-t, --tag" flag: invalid reference format

See 'docker build --help'.

Exited with code exit status 125

Solution

You'd expect the Docker command maybe being syntactically incorrect. But the error message is referring to something else: it turns out the git describe --tags command gave the fatal message.
The cause was that there was no git-tag set at all on the Github project yet.   After manually creating a release (including a tag) on Github and running the above build command again, the docker build command succeeded.