Git Submodule

2021-03-25 1620 words 8 minutes

Contents

When I added git submodule to my git repository and change that submodule during development, I encountered problem. I want to record how I fix that here.

1 Why I need to add submodule?

Because hugo requires submodule to reflect template.

1
2
3
4


hugo new site site
cd site
git init
git submodule add https://github.com/dillonzq/LoveIt.git themes/LoveIt

I could update this site theme to “LoveIt” (See Hugo LoveIt Theme).

2 Git structure and config

tree -ad command shows git related structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


$ cd site
$ tree -ad
├── .git
│   ├── hooks
│   ├── info
│   ├── logs
│   │   └── refs
│   │       ├── heads
│   │       └── remotes
│   │           └── origin
│   ├── modules
│   │   └── themes
│   │       └── LoveIt
│   │           ├── hooks
│   │           ├── info
│   │           ├── logs
│   │           │   └── refs
│   │           │       ├── heads
│   │           │       └── remotes
│   │           │           └── origin
│   ├── objects
│   │   ├── 00
...
│   ├── refs
│   │   ├── heads
...
├── static
└── themes
    └── LoveIt
        ├── .circleci
        ├── .github
        │   └── ISSUE_TEMPLATE
        ├── archetypes
        ├── assets
        │   ├── css
...

.git/config has following lines:

1
2
3


[submodule "themes/LoveIt"]
        url = https://github.com/dillonzq/LoveIt.git
        active = true

.gitmodules has following lines:

1
2
3


[submodule "themes/LoveIt"]
        path = themes/LoveIt 
        url = https://github.com/dillonzq/LoveIt.git 

submodule at .gitmodules:

path : submodule path at parent project
url : git URL for sub module

.gitmodules

If the others (including Gitlab-CI) build this parent project, they refer .gitmodules url and connect to the repository.

3 Change codes in submodule

3.1 Reflect chages in submodule directory

I fixed some problems at related in Hugo version compatibility. See Fix LoveIt and configuration

After that, checked git status at submodule directory.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


$ cd themes/LoveIt
$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified:   assets/css/_override.scss
modified:   layouts/partials/function/content.html

Untracked files:
(use "git add <file>..." to include in what will be committed)
i18n/jp.toml
layouts/shortcodes/instagram.html
no changes added to commit (use "git add" and/or "git commit -a")

Then commit it to local master branch at submodule directory.

1
2


$ git add .
$ git commit -m "Update for site customization"

Then git log says:

1
2
3
4
5
6
7


$ git log
commit 9ec7dd4dcbc8022e7930f6c515137db49bd71036 (HEAD -> master)
Author: Tatsuro Homma <tato.flam@gmail.com>
Date:   Tue Mar 23 16:58:49 2021 +0900

    Update for site customization 
...

9ec7dd4dcbc8022e7930f6c515137db49bd71036 is commit ID for this submodule.

Submodule referes commit ID

If the others (including Gitlab-CI) build this parent project, they refer .gitmodules url and connect to the repository.

3.2. Reflect changes in parent project

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


$ cd ../../
$ git status
On branch master
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified:   themes/LoveIt (new commits)

$ git diff
diff --git a/themes/LoveIt b/themes/LoveIt
index f787a4e..9ec7dd4 160000
--- a/themes/LoveIt
+++ b/themes/LoveIt
@@ -1 +1 @@
-Subproject commit f787a4e5ad4edf60467658d10c286248dc5027a6
+Subproject commit 9ec7dd4dcbc8022e7930f6c515137db49bd71036

Then updaste submodule to parent project.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


$ git add .
$ git status
On branch master
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
modified:   themes/LoveIt

$ git commit -m "Update submodule: themes/LoveIt for site customization"
[master da0cf1c] Update submodule: themes/LoveIt for site customization
1 file changed, 1 insertion(+), 1 deletion(-)

$ git status
On branch master
nothing to commit, working tree clean

git submodule update

Above git commit could to replaced by git submodule update command.

Ref:

4 Push project and build by Gitlab CI

Then I pushed parent project to Gitlab remote repository. but I have 2 kinds of error.

Git push error
Gitlab CI pipeline failed

4.1. git push error.

It’s not related to git submodule actually. That’s because I did extra ordinary procedure to setup remote repository with backup. What I did was:

git init local repository “site”
git add ., git commit -m ""
git remote add set-url <gitlab repository path> and git push local repository “site”
Change remote repository name and path(url) on gitlab to “site_clarity”
Change local repository name “site_clarity” and git remote set-url origin git@gitlab.com/tatofalm/site_clarity
git init local repository “site”
git add ., git commit -m ""
git remote add set-url <gitlab repository path> and git push local repository “site”

Original “site”(that was backed up as “site_clarity”) was for Hugo website template “Clarity” and new “site” is for template “LoveIt”. See: Hugo-LoveIt-Theme

Failure

git push says as follows.

1
2
3
4
5
6
7
8
9


$ git push -u origin master
To gitlab.com:tatoflam/site.git
! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'gitlab.com:tatoflam/site.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

I’m not sure about accurate reason for this, but it looks that git remote repository has records that are committed/pushed by former local repository even after renaming name and url path on Gitlab and configure right path by git remote set-url <path>.

The best way would be git pull and resolve merge conflict. However, it bothers to me as there are bunch of changing points with current Hugo template(LoveIt) from former one(Clarity). This is totally personal work and no one see this repository so far. That’s why I did git push -f origin master.

As Gitlab does not accept git push -f by default, I needed to change “Settings” > “Repository” > “Protected Branches” and activate “Allow force push” for Maintainers.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


$ git push -f origin master
Enumerating objects: 30, done.
Counting objects: 100% (30/30), done.
Delta compression using up to 8 threads
Compressing objects: 100% (19/19), done.
Writing objects: 100% (30/30), 78.88 KiB | 9.86 MiB/s, done.
Total 30 (delta 3), reused 0 (delta 0), pack-reused 0
remote:
remote: Project 'tatoflam/site' was moved to 'tatoflam/site_clarity'.
remote:
remote: Please update your Git remote:
remote:
remote: git remote set-url origin [git@gitlab.com](mailto:git@gitlab.com):tatoflam/site_clarity.git
remote:
remote:
To [gitlab.com](http://gitlab.com/):tatoflam/site.git
f1f2d2e...da0cf1c master -> master (forced update)

4.2. Gitlab CI pipeline failed

Though push completed, remote CI pipeline failed.

Gitlab > CI/CD > Jobs console says:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


Getting source from Git repository
00:04
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/tatoflam/site/.git/
Created fresh repository.
Checking out da0cf1cf as master...
Updating/initializing submodules recursively...
Submodule 'themes/LoveIt' (https://github.com/dillonzq/LoveIt.git) registered for path 'themes/LoveIt'
Cloning into '/builds/tatoflam/site/themes/LoveIt'...
fatal: remote error: upload-pack: not our ref 9ec7dd4dcbc8022e7930f6c515137db49bd71036
Fetched in submodule path 'themes/LoveIt', but it did not contain 9ec7dd4dcbc8022e7930f6c515137db49bd71036. Direct fetching of that commit failed.
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

It says it dit not contain commit ID 9ec7dd4dcbc8022e7930f6c515137db49bd71036 in submodule themes/LoveIt.

This is simply because I only committed it locally but not pushed it to remote repository. Gitlab CI/CD build refers external repository and could not find that commit.

As I tweaked theme (see Fix LoveIt and configuration), I needed to push it to remote folked repository and changed submodule path like:

Folked github dillonzq/LoveIt to my repository tatoflam/LoveIt.

Update remote (folked repository)

Set git remote url for themes/LoveIt

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


$ cd themes/LoveIt
$ git remote -v
origin  https://github.com/dillonzq/LoveIt.git (fetch)
origin  https://github.com/dillonzq/LoveIt.git (push)

$ git remote set-url origin git@github.com:tatoflam/LoveIt.git

$ git remote -v
origin  git@github.com:tatoflam/LoveIt.git (fetch)
origin  git@github.com:tatoflam/LoveIt.git (push)

Push submodule to folked repository
1

$ git push

Update submodule configuration.

Change url in .git/config

1
2


$ cd ../../
$ vi .git/config

1
2
3
4


[submodule "themes/LoveIt"]
    url = https://github.com/tatoflam/LoveIt.git
    active = true
    

Change url in .gitmodule

1
2
3
4


[submodule "themes/LoveIt"]
    path = themes/LoveIt
    url = https://github.com/tatoflam/LoveIt.git
    

Sync submodule

1
2
3
4
5


$ git submodule sync
Synchronizing submodule url for 'themes/LoveIt'

$ git submodule status
 9ec7dd4dcbc8022e7930f6c515137db49bd71036 themes/LoveIt (v0.2.10-1-g9ec7dd4)

Commit parent project

1
2
3
4


$ git add .
$ git commit -m "Sync submodule remote url"
[master 48df868] Sync submodule remote url
  1 file changed, 1 insertion(+), 1 deletion(-)

Push parent project
1

$ git push -u origin master

Then Gitlab CI build completed 😄

Bellow is a part of output by build job.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


Getting source from Git repository
00:04
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/tatoflam/site/.git/
Created fresh repository.
Checking out 48df868b as master...
Updating/initializing submodules recursively...
Submodule 'themes/LoveIt' (https://github.com/tatoflam/LoveIt.git) registered for path 'themes/LoveIt'
Cloning into '/builds/tatoflam/site/themes/LoveIt'...
Submodule path 'themes/LoveIt': checked out '9ec7dd4dcbc8022e7930f6c515137db49bd71036'
Entering 'themes/LoveIt'

Contents

Git Submodule

1 Why I need to add submodule?

2 Git structure and config

3 Change codes in submodule

3.1 Reflect chages in submodule directory

3.2. Reflect changes in parent project

4 Push project and build by Gitlab CI

4.1. git push error.

4.2. Gitlab CI pipeline failed

Reference