ConfigMap Auto Refresh, Spring Boot

Spring , JPA

ConfigMap Auto Refresh, Spring Boot

seulseul 2022. 12. 29. 21:49

선행사항
- kubernetes 설치 (kind , k3d , k3s ,k8s …)
- 해당 namespace 에 configmap <> sa role , rolebinding

아래 작업 수행시

ConfigMap 파일을 (= application.yml 내용) 수정한후 kubectl apply -f <configmap - 파일명> 을 해주면

Kubernetes Pod 재시작 없이 Spring Boot Application 에 수정한 설정이 적용된다.

1. Spring Boot Project

profile 은 dev 와 local 로 2개를 사용하고 있습니다.

dev	local
kubernetes 환경	local desktop IDE 실행환경

1.1. build.gradle

implementation 'org.springframework.cloud:spring-cloud-starter-kubernetes-fabric8-config'

해당 dependency 를 넣으면 spring app 에서 configmap 을 자동 reload 한다

1.2. bootstrap-dev.yml

bootstrap.yml 은 application.yml 보다 먼저 loading 되는 설정파일이다.
따라서 시스템 등급의 매개변수를 설정하면됩니다.
Spring Cloud Config Server 를 사용할때는 bootstrap.yml 에서 지정해야합니다.
그러나 Spring 2.4 이상의 버전부터 bootstrap.yml 을 사용하려면 dependency 추가가 필요합니다. (아래)

1.3. bootstrap dependency

// https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-bootstrap
implementation 'org.springframework.cloud:spring-cloud-starter-bootstrap'

2. Git Ops Project

2.1. kustomization.yaml

2.2 deployment.yaml

3. ArgoCD 에서 ConfigMap Apply

gitops 에서 application.yml 을 수정해서 커밋한 후
argocd 에서 configmap 을 apply only 로 sync 를 맞춰준다.
pod 재시작 없이 application 에서 바꾼 내용이 적용되는것을 확인할 수 있음

4. log 확인

4.1. application 시작

07:02:50.397 [restartedMain] WARN  o.s.c.k.f.config.Fabric8ConfigUtils - config-map with name : 'demo-scg-configmap-kubernetes' not present in namespace : 'gateway'
07:02:50.400 [restartedMain] WARN  o.s.c.k.f.config.Fabric8ConfigUtils - config-map with name : 'demo-scg-configmap-dev' not present in namespace : 'gateway'
07:02:50.401 [restartedMain] INFO  o.s.c.b.c.PropertySourceBootstrapConfiguration - Located property source: [BootstrapPropertySource {name='bootstrapProperties-configmap.demo-scg-configmap.gateway'}]
07:02:50.414 [restartedMain] INFO  com.demo.scg.DemoScgApplication - The following 2 profiles are active: "kubernetes", "dev"

application 이 시작할때 configmap 을 스캔하는 로그를 볼수 있다.

5. 개발 일 경우

5.1. skaffold with kubernetes

해당 config map 을 커밋즉시 바뀌는것이 아닌 즉각적으로 변경되는것을 확인하고 싶다면 GitOps 가 아닌 Project 에 deploy , service , configmap yaml 파일을 생성후

# skaffold 명령어
skaffold dev -p dev -t v3

config map 을 수정하면서 즉각적으로 변경되는것을 확인할 수 있습니다.

5.2. Local IDE 환경

인텔리제이나 이클리스 같은 ide 에서 Spring Boot Application 을 실행시킬 때는 Kubernetes Configmap 이 아닌 application.yml 을 설정파일로 사용하기때문에 spring.cloud.kubernetes 를 disable 처리해야한다.

spring application 시작시 bootstrap.yml 이 applicaiton.yml 보다 더 먼저 Scan 되므로 bootstrap 에서 enable : false 설정을 해준다.

bootstrap.yml

spring:
  profiles:
    active: local

profile 은 local 로 active 처리해둔다.

bootstrap-local.yml

# bootstrap-local.yml
spring:
  application:
    name: demo-scg
  cloud:
    kubernetes:
      enabled: false # enables all the sub-configurations
      reload:
        enabled: false
        monitoring-config-maps: false
      config:
        enabled: false

spring.cloud.kubernetes.enabled: false

로 처리해두면 IDE 실행시에도 ConfigMap 이 없다는 error message 가 뜨지 않게된다.

6. Trouble Shooting

configmap resource 를 read 할수 있는 권한이 있어야 하기 때문에 Role 과 RoleBinding 을 생성해줘야한다.

이는 namespace 별로 생성해야한다.

권한 확인

kubectl auth can-i get configmap --as=system:serviceaccount:gateway:default -n gateway

권한이 없을때 에러 메세지

06:03:01.128 [restartedMain] WARN  o.s.c.k.f.c.Fabric8ConfigMapPropertySource - Can't read configMap with name: [demo-scg-configmap] in namespace: [gateway]. Ignoring.
[demo-scg] io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://10.96.0.1/api/v1/namespaces/gateway/configmaps/demo-scg-configmap. Message: Forbidden!Configured service account doesn't have acce
ss. Service account may have been revoked. configmaps "demo-scg-configmap" is forbidden: User "system:serviceaccount:gateway:default"
cannot get resource "configmaps" in API group "" in the namespace "gateway".
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:697)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:676)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:627)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:566)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:527)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:494)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:476)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:705)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:191)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:158)
[demo-scg]      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:91)
[demo-scg]      at org.springframework.cloud.kubernetes.fabric8.config.Fabric8ConfigUtils.getConfigMapData(Fabric8ConfigUtils.java:118)
[demo-scg]      at org.springframework.cloud.kubernetes.fabric8.config.Fabric8ConfigMapPropertySource.getData(Fabric8ConfigMapPropertySource.java:79)
[demo-scg]      at org.springframework.cloud.kubernetes.fabric8.config.Fabric8ConfigMapPropertySource.<init>(Fabric8ConfigMapPropertySource.java:70)
[demo-scg]      at org.springframework.cloud.kubernetes.fabric8.config.Fabric8ConfigMapPropertySourceLocator.getMapPropertySource(Fabric8ConfigMapPropertySourceLocator.java:74)
[demo-scg]      at org.springframework.cloud.kubernetes.commons.config.ConfigMapPropertySourceLocator.getMapPropertySourceForSingleConfigMap(ConfigMapPropertySourceLocator.java:95)
[demo-scg]      at org.springframework.cloud.kubernetes.commons.config.ConfigMapPropertySourceLocator.lambda$locate$0(ConfigMapPropertySourceLocator.java:75)
[demo-scg]      at java.base/java.lang.Iterable.forEach(Unknown Source)
[demo-scg]      at org.springframework.cloud.kubernetes.commons.config.ConfigMapPropertySourceLocator.locate(ConfigMapPropertySourceLocator.java:75)
[demo-scg]      at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:51)
[demo-scg]      at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:47)
[demo-scg]      at org.springframework.cloud.kubernetes.commons.config.ConfigMapPropertySourceLocator.locateCollection(ConfigMapPropertySourceLocator.java:87)
[demo-scg]      at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:95)
[demo-scg]      at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:618)
[demo-scg]      at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:385)
[demo-scg]      at org.springframework.boot.SpringApplication.run(SpringApplication.java:306)
[demo-scg]      at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317)
[demo-scg]      at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
[demo-scg]      at com.demo.scg.DemoScgApplication.main(DemoScgApplication.java:13)
[demo-scg]      at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[demo-scg]      at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
[demo-scg]      at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
[demo-scg]      at java.base/java.lang.reflect.Method.invoke(Unknown Source)
[demo-scg]      at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
[demo-scg] 06:03:01.130 [restartedMain] INFO  o.s.c.b.c.PropertySourceBootstrapConfiguration - Located property source: [BootstrapPropertySource {name='bootstrapProperties-configmap.demo-scg-configmap.gateway'}]
[demo-scg] 06:03:01.148 [restartedMain] INFO  com.demo.scg.DemoScgApplication - The following 2 profiles are active: "kubernetes", "dev"
[demo-scg] 06:03:01.981 [restartedMain] INFO  o.s.cloud.context.scope.GenericScope - BeanFactory id=a99ac369-aab7-3b77-97f8-6372641a3f03
Watching for changes...
[demo-scg] 06:03:02.585 [restartedMain] INFO  o.h.validator.internal.util.Version - HV000001: Hibernate Validator 6.2.5.Final
[demo-scg] 06:03:02.875 [restartedMain] DEBUG o.s.c.g.config.GatewayProperties - Routes supplied from Gateway Properties: [RouteDefinition{id='demo-fe-board', predicates=[PredicateDefinition{name='Path', args={patterns=/boards/**}},
 PredicateDefinition{name='Method', args={methods.0=GET, methods.1=POST, methods.2=PUT, methods.3=DELETE}}], filters=[], uri=http://demo-fe-svc.fe.svc.cluster.local:9099, order=0, metadata={}}, RouteDefinition{id='demo-fe-api', pred
icates=[PredicateDefinition{name='Path', args={_genkey_0=/api/**}}], filters=[], uri=http://demo-fe-svc.fe:9099/, order=0, metadata={}}]
[demo-scg] 06:03:03.123 [restartedMain] INFO  c.demo.scg.config.Resilience4jConfig - >>>>>>>>>>>> START defaultCustomizer()
[demo-scg] 06:03:03.380 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [After]
[demo-scg] 06:03:03.380 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Before]
[demo-scg] 06:03:03.380 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Between]
[demo-scg] 06:03:03.380 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Cookie]
[demo-scg] 06:03:03.380 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Header]
[demo-scg] 06:03:03.380 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Host]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Method]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Path]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Query]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [ReadBody]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [RemoteAddr]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [XForwardedRemoteAddr]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [Weight]
[demo-scg] 06:03:03.381 [restartedMain] INFO  o.s.c.g.r.RouteDefinitionRouteLocator - Loaded RoutePredicateFactory [CloudFoundryRouteService]
Tags used in deployment:
 - jei0486/demo-scg -> jei0486/demo-scg:v3@sha256:d357b70161b0ccaac037d95798167569f961ee60c792daf16c18d19ee6d1b406
Starting deploy...
Loading images into kind cluster nodes...
Images loaded in 54ns
Waiting for deployments to stabilize...
 - gateway:deployment/demo-scg is ready.
Deployments stabilized in 1.054 second
Watching for changes...

Can't read configMap with name: [demo-scg-configmap] in namespace: [gateway]. Ignoring.
configmap 을 읽을수 없다는 error message 가 눈에 띈다.

7. 예외 사항

spring.datasource.url: jdbc:mariadb://localhost:3306/demo_board?characterEncoding=UTF-8

과 같은 DB Connection 관련 설정은 Pod 를 재시작해야 해당 설정이 적용된다.

왜냐하면, DB Connection 은 Spring 컨테이너가 시작할 때 Spring 컨텍스트의 초기화가 이뤄지며 생성되기 때문이다.

물론, ConfigMap 변경시 강제로 Refresh Bean 을 하는 방식의 Code 를 개발할 수는 있다.

그러나 이는, 시스템 장애 요인이 될 수 있기 때문에 지양하고, Pod 재시작하는 방법을 선택하는게 맞는것 같다.

참고

https://cloud.spring.io/spring-cloud-kubernetes/reference/html/#why-do-you-need-spring-cloud-kubernetes

Spring Cloud Kubernetes

In Kubernetes service registration is controlled by the platform, the application itself does not control registration as it may do in other platforms. For this reason using spring.cloud.service-registry.auto-registration.enabled or setting @EnableDiscover

cloud.spring.io

저작자표시

'Spring , JPA' 카테고리의 다른 글

[spring] Spring Webflux CRUD (0)	2023.01.16
[kafka] kafka Install (0)	2023.01.09
Webflux with MDC (0)	2023.01.04
[Spring Cloud Gateway] JWT & Opaque Token (0)	2023.01.04
@Slf4j 사용시 log cannot be resolved 에러 처리 (0)	2022.02.15

현재글ConfigMap Auto Refresh, Spring Boot

seulseul