Troubleshooting & Common Setup Fixes

Azure Admin Consent Fails: "Need admin approval"

AADSTS90094: Admin consent required

Root Cause: Your Microsoft Entra ID tenant restricts third-party application consent to Global Administrators or requires explicit admin consent workflows.

Resolution: Have an Entra ID Global Administrator or Privileged Role Administrator click "Connect with Microsoft" once in CostGlass to grant tenant-wide consent. Alternatively, your administrator can approve the CostGlass application in Microsoft Entra Admin Center -> Identity -> Applications -> Enterprise applications.

Azure Service Principal: "AuthorizationFailed / Does not have authorization"

AuthorizationFailed / 403 Forbidden

Root Cause: The Service Principal or Enterprise App was not assigned the required RBAC roles at the Subscription or Management Group scope.

Resolution: Run "./azure-templates/assign-costglass-roles.sh --app-id <COSTGLASS_APP_ID> --subscription <SUBSCRIPTION_ID>" or manually assign both "Reader" and "Cost Management Reader" roles to the CostGlass service principal in Azure Portal -> Subscriptions -> Access control (IAM).

Azure Client Secret Expired or Invalid

AADSTS7000215: Invalid client secret is provided

Root Cause: When using Bring-Your-Own Service Principal, the client secret created in Entra ID reached its expiration date or the secret ID was copied instead of the secret Value.

Resolution: In Microsoft Entra Admin Center -> App registrations -> Certificates & secrets, generate a new client secret, copy the "Value" (not Secret ID), and update the credentials in CostGlass Settings -> Cloud Accounts -> Azure.

OCI 401 NotAuthenticated: "The required information to complete authentication was not provided"

OCI 401 NotAuthenticated

Root Cause: The user OCID, tenancy OCID, region, or public key fingerprint does not match the key pair generated by CostGlass.

Resolution: In Oracle Cloud Console -> Identity -> Users -> CostGlassReadOnlyUser -> API Keys, confirm the fingerprint matches the fingerprint displayed on your CostGlass OCI onboarding page. Ensure the user OCID, tenancy OCID, and region match the Resource Manager stack outputs exactly with no leading/trailing spaces.

OCI Usage Reports ObjectNotFound or Access Denied

OCI 403/404 on usage-reports bucket

Root Cause: The IAM policy "endorse group CostGlassCostReaders to read objects in tenancy usage-report" was applied inside a sub-compartment rather than at the root tenancy level.

Resolution: OCI Usage Reports bucket is located in the root tenancy. Ensure the policy is defined in the root tenancy compartment (compartment_id = tenancy_ocid) rather than a child compartment.

Alibaba Cloud STS: "EntityNotExist.Role" or "NoPermission"

sts:AssumeRole / 403 Forbidden

Root Cause: The Role ARN entered in CostGlass does not exist, or the RAM role trust policy does not specify CostGlass's Alibaba Account ID and your unique External ID.

Resolution: In Alibaba Cloud RAM Console -> Roles -> CostGlassReadOnly -> Trust Policy, ensure the Principal is "acs:ram::<COSTGLASS_ALIBABA_ACCOUNT_ID>:root" and condition "sts:ExternalId" matches your CostGlass External ID. Confirm the Role ARN format is acs:ram::<account_id>:role/CostGlassReadOnly.

Alibaba Cloud OSS Billing Bucket Access Denied

NoSuchBucket / AccessDenied

Root Cause: The OSS bucket configured in Billing Management -> Bill Subscription does not match the BillingBucketName parameter entered when deploying the ROS template.

Resolution: Go to Alibaba Cloud Billing Management -> Bill Subscription, verify your target bucket name and region, and update the ROS stack parameter BillingBucketName to match the actual bucket name.

DigitalOcean 401 Unauthorized / Invalid Personal Access Token

HTTP 401 Unauthorized / DO API v2

Root Cause: The Personal Access Token was mistyped, revoked, or expired in the DigitalOcean Control Panel.

Resolution: In DigitalOcean Control Panel -> API -> Tokens/Keys, generate a new token with "Read" scope, copy the dop_v1_... string, and update the token in CostGlass Cloud Accounts -> DigitalOcean.

DigitalOcean Spend Displays Cached Figure

Local Cache TTL (≤30 min)

Root Cause: Live current period balance is cached for up to 30 minutes in browser storage to respect DigitalOcean API rate limits.

Resolution: Click "Refresh Now" next to the current period amount in CostGlass Cloud Accounts -> DigitalOcean to bypass the cache and fetch fresh balance data immediately.

Akamai / Linode 401 Unauthorized or 403 Forbidden

HTTP 401/403 Linode API v4

Root Cause: The Personal Access Token was created without the "account:read_only" scope or was revoked.

Resolution: In Linode Cloud Manager -> Profile -> API Tokens -> Create a Personal Access Token, ensure the Account scope is set to "Read Only" and all other scopes are set to "None". Paste the new token into CostGlass.

AWS STS AssumeRole "AccessDenied" (Role Cannot Be Assumed)

sts:AssumeRole / 403 Forbidden

Root Cause: The IAM Role Trust Policy does not specify the exact CostGlass Production Account ID (107489387542) or the sts:ExternalId condition is mismatched.

Resolution: Edit the role trust policy in the AWS IAM console. The Principal must be "arn:aws:iam::107489387542:root", and the StringEquals condition on sts:ExternalId must match the External ID shown on your Cloud Accounts page character for character — it is case-sensitive and a trailing space will fail. Also confirm the role is named exactly CostGlassReadOnly; any other name produces the same error.

Invalid ExternalId / StringEquals Condition Mismatch

sts:ExternalId mismatch

Root Cause: The External ID in the role trust policy does not match the one CostGlass sends. Usually a value typed by hand, copied from another tenant, or left as an example from a blog post.

Resolution: Open Cloud Accounts in CostGlass and use the copy button next to your External ID rather than retyping it, then update the StringEquals condition in the role trust policy to that exact value. If you deployed with CloudFormation, update the stack parameter instead of editing the role directly, or the next stack update will overwrite your change.

Connection fails with AccessDenied even though the role exists and the External ID is right

AccessDenied — wrong role name

Root Cause: The role is not named exactly CostGlassReadOnly. CostGlass is only permitted to assume that specific name, so a role called CostGlassReadOnlyRole, CostGlass-ReadOnly or anything else is refused before the trust policy is even considered.

Resolution: Rename the role to CostGlassReadOnly, or redeploy the CloudFormation template which sets the name for you. The ARN you paste into CostGlass should end in :role/CostGlassReadOnly.

Role assumption works but nothing appears for member accounts

partial organisation coverage

Root Cause: The role was created only in the management account. Cost Explorer in the management account covers consolidated billing, but resource discovery has to run inside each member account.

Resolution: Deploy the AWS Organization template rather than the single-account one. It provisions the same role across every member account via StackSets, and auto-deployment covers accounts added later.

StackSet Operation Fails: Account is not an active Organization member

AccountNotActiveException

Root Cause: An AWS account suspended, closed, or pending invitation exists inside the AWS Organization OU hierarchy.

Resolution: In CloudFormation StackSets, set "FailureToleranceCount" to a non-zero value or explicitly target active Organization Units (OUs) excluding suspended sandbox accounts.

CloudFormation ValidationError: Role "CostGlassReadOnly" already exists

AlreadyExistsException

Root Cause: An IAM role named CostGlassReadOnly was previously created manually or by a prior stack in that member account.

Resolution: Delete the existing legacy IAM role in the target member account or choose a custom role name parameter in the StackSet template.

Terraform Error: User is not authorized to perform: iam:CreateRole

AccessDeniedException / Terraform apply

Root Cause: The AWS credential running terraform apply lacks IAM permission boundaries or AdministratorAccess.

Resolution: Ensure your deployer IAM role has permissions for "iam:CreateRole", "iam:AttachRolePolicy", and "iam:PutRolePolicy".

Terraform Provider Inconsistent Final Plan on Multi-Region KMS

Provider produced inconsistent final plan

Root Cause: Cross-region KMS multi-region replica keys created asynchronously before AWS IAM propagates the trust key policy.

Resolution: Add an explicit depends_on = [aws_kms_key.primary] block and introduce a 15-second time_sleep resource before attaching the replica key policy.

Kubernetes API 401 Unauthorized / Expired Kubeconfig Token

HTTP 401 Unauthorized / k8s api

Root Cause: The client Kubeconfig uses a short-lived AWS IAM Authenticator token (aws-iam-authenticator or aws eks get-token) which expires after 15 minutes.

Resolution: Create a long-lived ServiceAccount token using the read-only RBAC manifest in the Managing cloud accounts guide, and use that instead of a kubeconfig that depends on aws eks get-token. Cost allocation does not need this at all — the OpenCost exporter authenticates with its own ingest token and is unaffected.

OpenCost reports every workload as costing zero

MetricsServerNotAvailable / no pricing

Root Cause: OpenCost needs the Kubernetes metrics server to see CPU and memory usage. Managed EKS clusters do not install it by default, and without it OpenCost has nothing to apportion node cost against.

Resolution: Install the metrics server, then give OpenCost a few minutes to build a window: kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Exporter CronJob runs but CostGlass shows no data for the cluster

exporter push not received

Root Cause: Either the job cannot reach OpenCost inside the cluster, or it cannot reach the CostGlass API on the way out. The job log distinguishes the two in one line.

Resolution: Run the job immediately rather than waiting for the schedule: kubectl create job -n opencost --from=cronjob/costglass-opencost-exporter debug-run, then kubectl logs -n opencost job/debug-run. A connection error to opencost.opencost.svc means the service name or namespace differs; a timeout on the POST means outbound HTTPS is blocked from that namespace.

ImagePullBackOff on the costglass-opencost-exporter pod

ImagePullBackOff / ErrImagePull

Root Cause: The nodes cannot pull public images — an air-gapped cluster, a blocked registry, or a missing imagePullSecret.

Resolution: Mirror python:3.11-alpine into your own ECR registry and change the image reference in the CronJob manifest to the internal path.

Ingest token was committed to Git or shared by mistake

X-CostGlass-K8s-Token exposure

Root Cause: The token authorises writing cost data for one cluster. It grants no read access to your data and cannot reach anything else, but it should still be rotated.

Resolution: Regenerate the token from FinOps -> Kubernetes for that cluster. The previous value stops working immediately. Store the new one in a Kubernetes Secret and reference it from the CronJob rather than inlining it in the manifest.

Cost Explorer ce:GetCostAndUsage Disabled on Member Account

AccessDeniedException: ce:GetCostAndUsage

Root Cause: AWS Cost Explorer access is restricted by default in linked member accounts if Cost Management preferences are locked by the Payer.

Resolution: In the AWS Management/Payer account, go to Billing Console -> Cost Management Preferences -> Linked Account Access and check "Cost Explorer Access: Enabled".

S3 Access Denied on CUR 2.0 Parquet Export Bucket

s3:GetObject / AccessDenied

Root Cause: The S3 bucket containing billing exports is encrypted with an AWS KMS Customer Managed Key (CMK) whose key policy omits CostGlass.

Resolution: Add the CostGlass IAM Role ARN to the KMS Key Policy Statement for "kms:Decrypt" and "kms:DescribeKey" under AWS KMS.

Spend Data Appears 24–48 Hours Delayed

AWS Billing Latency (Working As Designed)

Root Cause: AWS Cost & Usage Reports (CUR 2.0) are generated by Amazon at 8-to-24 hour intervals. Final consolidated billing data is updated incrementally.

Resolution: CostGlass provides real-time Kubernetes and CloudWatch telemetry for intraday usage, while historical dollar amounts reconcile automatically upon AWS CUR delivery.

Snowflake says it could not query ORGANIZATION_USAGE

Could not query ORGANIZATION_USAGE

Root Cause: CostGlass reads SNOWFLAKE.ORGANIZATION_USAGE.USAGE_IN_CURRENCY_DAILY, and that view is not available to ordinary roles. The role the CostGlass user connects with has not been given access to it by an ORGADMIN.

Resolution: Ask your Snowflake organisation administrator (ORGADMIN) to grant that role IMPORTED PRIVILEGES on the SNOWFLAKE database, then click Sync on the Snowflake card. CostGlass does not fall back to an estimate.

Datadog 403 Forbidden / Invalid Application Key Scope

HTTP 403 Forbidden / Datadog API

Root Cause: The Datadog Application Key lacks the required "usage_read" scope permission in the Datadog Organization settings.

Resolution: In Datadog Console -> Organization Settings -> Application Keys, edit your key and check the "usage_read" permission scope.

Snowflake Error: Database "SNOWFLAKE" Does Not Exist or Not Authorized

SQL compilation error: 002003

Root Cause: The CostGlass reader role was not granted imported privileges on the shared SNOWFLAKE analytical database.

Resolution: Run the setup SQL command as ACCOUNTADMIN: GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE TO ROLE COSTGLASS_FINOPS_READER;

Virtual Tag Rules Created But Historical Costs Not Updating Immediately

2D Matrix Cache Refresh Delay

Root Cause: Analytical 2D Matrix aggregations are cached with a 15-minute TTL to ensure fast page loads.

Resolution: Click the "Force Refresh" button at the top right of the 2D Cost Matrix to bypass the DuckDB analytical cache immediately.