17 Commits

Author SHA1 Message Date
e24ef4e2d8 New draft post and add menu 2025-06-02 22:38:43 -04:00
555e1f57d5 add tags 2025-05-29 18:25:13 -04:00
954536602f Update theme and new make targets 2025-05-29 11:18:52 -04:00
1645042922 Add fedi link
Link needs to contain rel="me" for verification
2025-05-28 20:08:58 -04:00
6c0ad94800 regen and update Makefile 2025-05-28 20:05:38 -04:00
88ab4756e4 Update home page and asciinema poster
Also remove trailing whitespace in all files
2025-05-28 17:31:56 -04:00
67bdd8b49a make casts local 2025-05-28 17:21:41 -04:00
149518e65e change to theme 2025-05-28 17:15:43 -04:00
794723d726 make build clean 2025-05-20 23:17:10 -04:00
e8bce84bf8 update existing 2025-05-20 23:13:03 -04:00
f3c98b3e46 add shortcode for collapsable section 2024-07-05 02:05:55 -04:00
d2a3d53e54 Update asciinema and hugo version 2024-06-27 10:02:29 -04:00
425dd6cdd2 update base url 2024-04-19 17:45:30 -04:00
6205d19bbb Add new post unix-sockets-perms 2024-04-19 17:40:49 -04:00
83e8868495 Add new post unix-sockets-perms 2024-04-19 17:38:40 -04:00
ffd7b06cc0 update build 2024-04-18 20:54:29 -04:00
88aa2e8d11 update deps 2024-04-18 20:46:41 -04:00
26 changed files with 1456 additions and 1582 deletions

1
.gitignore vendored
View File

@@ -1 +1,2 @@
.hugo_build.lock
tags

3
.gitmodules vendored
View File

@@ -1,3 +1,6 @@
[submodule "public"]
path = public
url = https://gitea.balki.me/balki/blog-op
[submodule "themes/mytheme"]
path = themes/mytheme
url = https://gitea.balki.me/balki/no-js-hugo-theme.git

View File

@@ -1,21 +1,20 @@
ASCIINEMA_VERSION=3.5.0
.PHONY: update-bootstrap
update-bootstrap:
curl -L "https://cdn.jsdelivr.net/npm/bootstrap@5/dist/css/bootstrap.min.css" -o static/assets/bootstrap.min.css
curl -L "https://cdn.jsdelivr.net/npm/bootstrap@5/dist/js/bootstrap.min.js" -o static/assets/bootstrap.min.js
.PHONY: update-asciinema
update-asciinema:
curl -L "https://github.com/asciinema/asciinema-player/releases/download/v$(ASCIINEMA_VERSION)/asciinema-player.css" -o static/assets/asciinema-player.css
curl -L "https://github.com/asciinema/asciinema-player/releases/download/v$(ASCIINEMA_VERSION)/asciinema-player.min.js" -o static/assets/asciinema-player.min.js
.PHONY: pulltheme
pulltheme:
git -C themes/mytheme pull
.PHONY: deploy
deploy:
rsync -av public desk:/var/www/balki.me
git -C public push
ssh desk git -C /var/www/balki.me/public pull
.PHONY: build
build:
rm -rf public/*
hugo --logLevel debug
.PHONY: serve
serve:
hugo server -D --bind "0.0.0.0" --navigateToChanged
hugo server -D --bind "0.0.0.0" --navigateToChanged -p 2323
tags:
~/projects/mine/hugotags/hugotags.sh ~/projects/notmine/hugoDocs/content/en > tags

View File

@@ -1,3 +0,0 @@
baseURL: http://balki.me/
languageCode: en-us
title: Balki's Blog

View File

@@ -1,9 +1,11 @@
---
title: "Home"
date: 2023-05-31T09:14:56-04:00
draft: false
---
Balki's Blog
============
Just another dev blog.
* git: https://gitea.balki.me/balki
* github: https://github.com/balki
* Fediverse: {{< myfedilink >}} `@balki@balki.me`
* RSS: https://blog.balki.me/index.xml

View File

@@ -10,3 +10,26 @@ asciinema: true
{{< asciinema key="demo" >}}
There should be a asciinema cast above!
{{< details title="Click to see sample json" >}}
```json
{
"external_tracker_url": "https://github.com/caddyserver/caddy/issues",
"external_tracker_format": "https://github.com/caddyserver/caddy/issues/{index}",
"external_tracker_style": "numeric",
"external_tracker_regexp_pattern": ""
}
```
{{< /details >}}
SWBrqK9fAC6XuZ7eUl3xwnW8SOfekv01TntKdNl/0+noC4Xel+A2elveBYSxh4gWBtMqXR0mT7+8
N1jVF/o/Q19m8JF8UC9SiJP01CGDJoROcN2ijdyguLg76A0B2tltU+eqlbm46Pml+Ukndda1wxdY
eWs1favc/qsN2txx5jL/6srrEEVIKr3p6ov4hOVtju0dswaseknxZagMfaBwbk6KNGS9r5Nmk+MY
0jdYOfcBIHGB/IxvifV+ux5l/C7VK/PBFOm6t/qtVbczDiPtzngOcsR/CQOej5Qfi+2mXEVbci53
GMois/cjJF4vSU7/OcdxCkkLpN42UKyFOPiR3TczXrsM0kt1Zl/y8huf9KLkRC61MD2Zn/FOvl4U
Jsd9CsVNqN2DO5ABxHqSJ5oI9d03rJilWFunJ6jURH8CyfQv5vJ8ORQmMZdBF1DaeXm6K7/R26lY
ArRpYtgWNhzDWhyGSM2E3/XGvE1w4iZ/u8ml8K6R+tInxYRd1+6ScdTr0NgweuXbnk/27G66+NVv
LX3t9d+rWvwdJBzibimLzy6Rnxq6XO1CSmBVTagr4edyJe76GqSS2RMhk3zbdUyP5gyPrYyCDQn5
/0rQsHmzGd9izC6Gjuk7hu10Y3eWi5DCfrDGfyrADO90uYqXrW2tjliwdRJuZtSuRxbOVaY3G3UX
MPG6pRH/byyKTduN2RS4VNo73ZkQOOooQ3NQErGPc29yeWdvLoA7DB5rsMf7V0XZcpLrUVM6SoQf
<!-- vim: set lw=DETAILS_FDM: -->

View File

@@ -0,0 +1,209 @@
---
title: "Mirroring git repositories with Gitea"
date: 2024-06-27T10:05:46-04:00
draft: true
tags:
- gitea
- curl
- yq
categories:
- development
---
[Gitea][1] is an awesome self hosted git forge. I use the [pull-mirror][2]
feature to mirror many git repos (mostly from github). In this post, I want to
share few maintenance scripts I ran that connects to [gitea-api][14] using
[yq][] and [curl][].
<!--more-->
# TODO
* Change mirror interval
* Disable for mirror repos by default actions
* use jo https://github.com/jpmens/jo
* get just latest mirror and apply the settings
## Setup
### API token
Go to `https://gitea.balki.me/user/settings/applications` and Generate new
Token with write permission for repositories.
# TODO
set base url
## Why Mirror?
1. Remote sites may disappear one day.
2. Better local code search. Github does not allow code search without signing
in.
3. Tools like vim-plugins can use the local urls which is better for privacy.
4. Setup notification when a repository creates a new tag.
## Dumb crawlers problem
My gitea [instance][3] is public and had all mirror repos public as well. This
caused a huge network traffic from bots.
I created an [organization][4] without public visibility and made it own all
the mirror repos.
```
yq --version
yq (https://github.com/mikefarah/yq/) version v4.44.1
curl -V | head -c 11
curl 8.8.0
```
## API token
## Hooks for notification
[Hooks][5]
Lets first download the list of all mirror repos
```bash
TOKEN=d88446542e844f4da4ba75bbb85bd694a71907b5
curl "https://gitea.balki.me/api/v1/repos/search?limit=100&mode=mirror" \
-H "accept: application/json" \
-H "Authorization: token $TOKEN" \
-o mirror-repos.json
```
References
1. API doc: https://gitea.balki.me/api/swagger#/repository/repoSearch
Create a [hook][5] manually in one repo and get the hook using the API
```bash
curl -s "https://gitea.balki.me/api/v1/repos/MirrorWatch/snac2/hooks?page=1&limit=10" \
-H "Authorization: token $TOKEN" | yq -P -oj
```
Sample Output:
```json
[
{
"id": 32,
"type": "telegram",
"branch_filter": "tag",
"config": {
"content_type": "json",
"url": "https://api.telegram.org/bot1169894068:J1JVbV3f2vEQpdnPqFANfhjWZrFuUCJs1EW/sendMessage?chat_id=-1008910751069"
},
"events": [
"create"
],
"authorization_header": "",
"active": true,
"updated_at": "2024-06-20T20:54:33-04:00",
"created_at": "2024-06-20T20:54:33-04:00"
}
]
```
Now loop through all mirror repos and add the same webook. Remove unwanted fields like `id`, `created_at`, etc.,
```bash
yq -r '.data[] | .full_name' mirror-repos.json | while read -r repo; do
echo "$repo"
curl "https://gitea.balki.me/api/v1/repos/$repo/hooks" \
-H "Authorization: token $TOKEN" \
--json @- <<-EOM
{
"active": true,
"branch_filter": "tag",
"config": {
"content_type": "json",
"url": "https://api.telegram.org/bot1169894068:J1JVbV3f2vEQpdnPqFANfhjWZrFuUCJs1EW/sendMessage?chat_id=-1008910751069"
},
"events": [
"create"
],
"type": "telegram"
}
EOM
echo "============"
done
```
## Fixing issues and pr links
Fix the issue url setting in first repo as shown [here][7].
Get the json representataion.
```bash
yq '.data[] | .external_tracker ' mirror-repos.json | head
```
Sample output
```json
{
"external_tracker_url": "https://github.com/caddyserver/caddy/issues",
"external_tracker_format": "https://github.com/caddyserver/caddy/issues/{index}",
"external_tracker_style": "numeric",
"external_tracker_regexp_pattern": ""
}
```
Now loop throug all repos and update. Making sure only add to github repos and
they are not already updated
```bash
yq -r '.data[]
| select(.original_url == "*github*" and has("internal_tracker") )
| "\(.full_name) \(.original_url)"' mirror-repos.json | while read -r repo og; do
echo "Repo is $repo and github origin url is $og"
curl "https://gitea.balki.me/api/v1/repos/$repo" \
-H "Authorization: token $TOKEN" \
-X PATCH \
--json @- <<-EOM
{
"external_tracker": {
"external_tracker_url": "${og%.git}/issues",
"external_tracker_format": "${og%.git}/issues/{index}",
"external_tracker_style": "numeric",
"external_tracker_regexp_pattern": ""
}
}
EOM
done
yq -r '.data[] | .full_name' mirror-repos.json | while read -r repo; do
echo "$repo"
jo has_actions=false | curl "https://gitea.balki.me/api/v1/repos/$repo" \
-H "Authorization: token $TOKEN" \
-X PATCH \
--json @-
done
```
### Doc links
* yq: [select][8], [has][9], [string interpolation][10]
* bash: [parameter expansion][11], [here-doc][12]
* curl: [`--json`][13]
[1]: https://github.com/go-gitea/gitea
[2]: https://docs.gitea.com/usage/repo-mirror#pulling-from-a-remote-repository
[3]: https://gitea.balki.me
[4]: https://docs.gitea.com/usage/permissions#organization-repository
[5]: https://docs.gitea.com/usage/webhooks
[6]: https://docs.gitea.com/development/api-usage
[7]: https://github.com/go-gitea/gitea/issues/18986
[8]: https://mikefarah.gitbook.io/yq/operators/select
[9]: https://mikefarah.gitbook.io/yq/operators/has#select-checking-for-existence-of-deep-paths
[10]: https://mikefarah.gitbook.io/yq/operators/string-operators#interpolation
[11]: https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
[12]: https://www.gnu.org/software/bash/manual/html_node/Redirections.html#Here-Documents
[13]: https://everything.curl.dev/http/post/json.html
[14]: https://gitea.balki.me/api/swagger#/repository/repoSearch

View File

@@ -0,0 +1,7 @@
---
title: "Revive Blog"
date: 2025-05-29T18:26:59-04:00
draft: true
---
{{% gitea repo=balki/hugotags path=hugotags.sh lang=bash %}}

View File

@@ -0,0 +1,74 @@
---
title: "Securing communication between webserver and app"
date: 2024-04-19T17:31:06-04:00
tags:
- unix-sockets
- tmpfiles.d
- webserver
categories:
- sysadmin
---
Webapps usually listen on a random tcp port and a web server forwards the
requests to it. Webserver handles tls, static asset serving and sometimes
authentication, bruteforce check etc., However any local user in the system can
directly connect to the app's listen port bypassing the web server and thus
loose the protections offered by the webserver.
### Unix sockets
Unix sockets are special files in filesystem that processes can use to
communicate instead of tcp ports. Since they are files, filesytem ownership and
permissions can be used to restrict which system-users can listen or connect to
them
### Caddy ↔ Gitea
I use [Caddy][1] web server which reverse-proxies to [Gitea][2] server. For
caddy to connect to gitea and also disallow anyother user to connect, we want a
socket like below
```bash
srw-rw---- 1 gitea caddy 0 Apr 17 21:24 /run/gitea.sock
```
Unfortunately neither `user caddy` nor `user gitea`, can create such socket.
Regular users can only create files owned by themselves. Only root can
create/change ownership of files and folders.
### `tmpfiles.d` to rescue
[tmpfiles.d][3] provides a way to do it. Since it is run as root, it can create
files and directories as any user.
```bash
cat /etc/tmpfiles.d/caddy-run-unix.conf
d /run/gitea-caddy 0750 gitea caddy -
```
Above config creates below directory every time on startup.
```bash
sudo ls -ld /run/gitea-caddy/
drwxr-x--- 2 gitea caddy 60 Apr 17 21:24 /run/gitea-caddy/
```
With those permissions and ownership, only `user gitea` can create the socket
in `/run/gitea-caddy` and only `user caddy` can `cd` into that directory.
```bash
sudo ls -l /run/gitea-caddy/web.sock
srw-rw-rw- 1 gitea gitea 0 Apr 17 21:24 /run/gitea-caddy/web.sock
```
The socket file permission can be more liberal as no other user can read into
the `/run/gitea-caddy` directory
### Bonus!
If your app does not connect to any external services, it can even be run in a
[private network][4].
[1]: https://caddyserver.com
[2]: https://gitea.com
[3]: https://man.archlinux.org/man/tmpfiles.d.5
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#PrivateNetwork=

View File

@@ -101,7 +101,7 @@ autocmd BufNewFile,BufRead cheat.sh nnoremap <buffer> <silent><2-LeftMouse> :Sen
### Demo
{{< asciinema key="vimstt" >}}
{{< asciinema key="vimstt" poster="npt:0:23" startAt="0:18" >}}
### Using

32
hugo.yaml Normal file
View File

@@ -0,0 +1,32 @@
baseURL: https://blog.balki.me/
languageCode: en-us
title: Balki's Blog
theme: mytheme
markup:
highlight:
# This setting makes code highlighting use theme's colors instead of hugo's default
noClasses: false
params:
themeStyleSwitcher: true
footerText: "[theme](https://gitea.balki.me/balki/no-js-hugo-theme)"
showTagList: false
menu:
main:
- identifier: home
name: Home
pageRef: /
weight: 1
- identifier: gitea
name: Gitea
url: https://gitea.balki.me/balki
weight: 2
- identifier: github
name: Github
url: https://github.com/balki
weight: 3
- identifier: rss
name: RSS
url: https://blog.balki.me/index.xml
weight: 4

View File

@@ -1,58 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="Content-Security-Policy" content="default-src 'self' 'unsafe-inline' data: ;{{ if .Params.asciinema }}script-src 'self' 'unsafe-inline' 'unsafe-eval';{{ end }}">
<link rel="alternate" type="application/rss+xml" href="/posts/index.xml">
<title>
{{- block "title" . -}}
{{ .Site.Title }}
{{- end -}}
</title>
<!-- Diable favicon requests: https://stackoverflow.com/a/13416784 -->
<link rel="icon" href="data:;base64,iVBORw0KGgo=">
<link href="/assets/bootstrap.min.css" rel="stylesheet" >
{{ if .Params.asciinema }}
<link rel="stylesheet" type="text/css" href="/assets/asciinema-player.css" />
{{ end }}
<style>
{{ block "styles" . }}
{{ end }}
div.highlight pre {
{{/* code highlighting */}}
padding: 1rem;
}
</style>
<script src="/assets/bootstrap.min.js" defer></script>
</head>
<body>
<div class="container" >
<aside>
<nav>
<ul class="nav">
<li class="nav-item"><a class="nav-link" href="/"> Home </a></li>
<li class="nav-item"><a class="nav-link" href="/posts"> Blog </a></li>
<li class="nav-item"><a class="nav-link" href="https://gitea.balki.me/balki"> Gitea </a></li>
<li class="nav-item"><a class="nav-link" href="https://github.com/balki"> Github </a></li>
<!-- TODO
<li class="nav-item"><a class="nav-link" href="/about"> About </a></li>
-->
</ul>
</nav>
</aside>
<main>
{{ block "main" . }}
{{ end }}
</main>
<hr>
<div style="height: 300px;"><!-- This space is intentionally left blank --></div>
</div>
</body>
</html>

View File

@@ -1,6 +0,0 @@
{{ define "main" }}
<h1 class="text-center">Posts</h1>
{{ range .Pages }}
{{ partial "postcard" . }}
{{ end }}
{{ end }}

View File

@@ -1,24 +0,0 @@
{{ define "title" }}
{{ .Title }} &ndash; {{ .Site.Title }}
{{ end }}
{{ define "main" }}
<article>
<h1 class="display-3 text-center">{{ .Title }}</h1>
<p class="text-center"><time datetime="{{ .Date }}">{{ .Date.Format "2006-01-02" }}</time>
{{ if .Params.tags }}
{{range .Params.tags}}
<a class="btn btn-primary" href="/tags/{{ . | urlize }}" role="button">{{ . }}</a>
{{end}}
{{end}}
{{ if .Params.categories }}
{{range .Params.categories}}
<a class="btn btn-secondary" href="/categories/{{ . | urlize }}" role="button">{{ . }}</a>
{{end}}
{{end}}
</p>
{{ .Content }}
</article>
{{ end }}

View File

@@ -0,0 +1,12 @@
{{- $url := "" -}}
{{- with .Get "url" -}}
{{- $url = . -}}
{{- else -}}
{{- $url = urls.JoinPath "http://gitea.balki.me" (.Get "repo") "raw/branch/main" (.Get "path") -}}
{{- end -}}
```{{ .Get "lang" }}
{{ with resources.GetRemote $url }}
{{- .Content -}}
{{ end }}
```
[source]({{ $url }})

View File

@@ -0,0 +1 @@
<a href="https://social.balki.me/@balki" rel="me" >https://social.balki.me/@balki</a>

View File

@@ -1,16 +0,0 @@
{{ define "main" }}
<article>
<div class="text-center m-4">
{{ .Content }}
</div>
<p class="text-center">Under construction. Come back later! :)</p>
<div>
<h4>Recent posts</h4>
{{ range .Pages }}
{{ range .Pages }}
{{ partial "postcard" . }}
{{ end }}
{{ end }}
</div>
</article>
{{ end }}

View File

@@ -1,12 +0,0 @@
{{ define "styles" }}
a.title-link {
text-decoration: none;
}
{{ end }}
<div class="card mb-4">
<div class="card-body">
<a class="title-link link-primary" href="{{ .RelPermalink }}" ><h3 class="card-title">{{ .Title }}</h3></a>
<p> {{ .Date.Format "2006-01-02" }} </p>
<p class="card-text"> {{ .Summary }} </p>
</div>
</div>

View File

@@ -1,20 +0,0 @@
<div id='{{ with .Get "key" }}{{ . }}{{ end }}'></div>
<script src="/assets/asciinema-player.min.js"></script>
<script>
AsciinemaPlayer.create('/casts/{{ with .Get "key" }}{{ . }}{{ end }}.cast', document.getElementById('{{ with .Get "key" }}{{ . }}{{ end }}'), {
{{ if .Get "cols" }}cols: "{{ .Get "cols" }}",{{ end }}
{{ if .Get "rows" }}rows: "{{ .Get "rows" }}",{{ end }}
{{ if .Get "autoPlay" }}autoPlay: "{{ .Get "autoPlay" }}",{{ end }}
{{ if .Get "preload" }}preload: "{{ .Get "preload" }}",{{ end }}
{{ if .Get "loop" }}loop: "{{ .Get "loop" }}",{{ end }}
{{ if .Get "startAt" }}startAt: "{{ .Get "startAt" }}",{{ end }}
{{ if .Get "speed" }}speed: "{{ .Get "speed" }}",{{ end }}
{{ if .Get "idleTimeLimit" }}idleTimeLimit: "{{ .Get "idleTimeLimit" }}",{{ end }}
{{ if .Get "theme" }}theme: "{{ .Get "theme" }}",{{ end }}
{{ if .Get "poster" }}poster: "{{ .Get "poster" }}",{{ end }}
{{ if .Get "fit" }}fit: "{{ .Get "fit" }}",{{ end }}
{{ if .Get "terminalFontSize" }}terminalFontSize: "{{ .Get "terminalFontSize" }}",{{ end }}
{{ if .Get "terminalFontFamily" }}terminalFontFamily: "{{ .Get "terminalFontFamily" }}",{{ end }}
{{ if .Get "terminalLineHeight" }}terminalLineHeight: "{{ .Get "terminalLineHeight" }}",{{ end }}
});
</script>

2
public

Submodule public updated: 6b4eafe671...e03e9e4c2c

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

1
themes/mytheme Submodule

Submodule themes/mytheme added at da512608d9