TL;DR

1

https://grafana.example.com/login?disableAutoLogin=true

That takes you to the username and password form even when auto_login = true keeps redirecting you to the OAuth provider. If you also do not remember the local admin password:

1
2
3
4

# container / pod
grafana cli --homepath /usr/share/grafana admin reset-admin-password 'new'
# package
grafana-cli admin reset-admin-password 'new'

And if the form is disabled ([auth] disable_login_form = true), first set GF_AUTH_DISABLE_LOGIN_FORM=false in the environment and restart.

Roles that undo themselves

You give someone Editor in Administration → Users, and the next day they are Viewer again. Not a bug: on every OAuth login Grafana syncs the organization role with whatever the provider says (or with auto_assign_org_role when the provider says nothing) and overwrites what you set by hand.

Two ways to live with it:

1. Let the provider really be in charge. Map groups or claims to roles with JMESPath and make it strict, so a user with no group does not get in as a default Viewer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

[auth.github]
enabled = true
allow_sign_up = true
auto_login = true
client_id = $__file{/etc/secrets/github-client-id}
client_secret = $__file{/etc/secrets/github-client-secret}
scopes = user:email,read:org
allowed_organizations = my-org
role_attribute_strict = true
allow_assign_grafana_admin = true
role_attribute_path = contains(groups[*], '@my-org/grafana-admin') && 'GrafanaAdmin' || contains(groups[*], '@my-org/grafana-editor') && 'Editor' || contains(groups[*], '@my-org/grafana-viewer') && 'Viewer'

2. Keep Grafana away from the roles. If you would rather manage them in the UI:

1
2

[auth.github]
skip_org_role_sync = true

skip_org_role_sync is per provider (auth.github, auth.google, auth.generic_oauth, auth.azuread, auth.okta, auth.gitlab, SAML). The old global switch [auth] oauth_skip_org_role_update_sync is deprecated; if it is still in your .ini, replace it with the per-provider one.

With Google, hosted_domain and allowed_domains restrict who can log in; without role_attribute_path everyone comes in with auto_assign_org_role (Viewer by default).

Provisioned data sources

Data sources defined in provisioning/datasources/*.yaml (in the Helm chart, datasources.datasources.yaml) have three surprising properties:

  • They cannot be edited in the UI when they carry editable: false: the form shows up locked. That is the right thing in production; the change goes in the YAML.
  • Secrets go in secureJsonData, not in jsonData: Grafana encrypts them with secret_key before storing them in its database (/var/lib/grafana/grafana.db, SQLite by default) and never returns them through the API. Tokens, passwords and client certificates go there; in jsonData they sit in plain text.
  • They do not disappear when removed from the YAML. Grafana only inserts and updates; deleting has to be explicit:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

apiVersion: 1
prune: true                  # recent versions: removes provisioned sources no longer in this file
deleteDatasources:
  - name: Sentry-old
    orgId: 1
datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    orgId: 1
    url: http://prometheus-server.monitoring.svc.cluster.local
    isDefault: true
    editable: false
    version: 1
  - name: Sentry
    type: grafana-sentry-datasource
    access: proxy
    orgId: 1
    editable: false
    jsonData:
      url: https://sentry.io
      orgSlug: my-org
    secureJsonData:
      authToken: ${SENTRY_TOKEN}      # environment variable of the Grafana process

Environment variables expand with $VAR or ${VAR}; a literal $ in a secret is written $$.

Traps

  • auto_login with several providers is ignored: it only works with a single provider configured for auto-login.
  • allow_sign_up = false keeps out any new user from the provider, including the one you just added to the group; the log says “user signup disabled”.
  • role_attribute_strict = true with no mapped group rejects the login with “Role attribute not found”; that is what you want, but warn people.
  • The local admin still exists with SSO: change its password and keep it in the secret manager, it is your emergency door together with disableAutoLogin.
  • The default secret_key (SW2YcwTIb9zpOOhoPsMm) encrypts secureJsonData identically on every Grafana; set your own ([security] secret_key) before provisioning anything, because changing it later invalidates what was encrypted.

References: Configure authentication and Provisioning in the Grafana documentation.

See also