来源：客响 ## 问题背景 ![](../../../../attachment/images-paste/image-20240118131714024.png) ## 问题详情 湖北移动（cache+ 企业） 自助配置域名个数：208 个 ``` 已处理：pme58e96a1bcc642a8b54c0abcab823e14,10个域名（存在） 已处理：pmc32b4c985b3f4816a833c863241e23f4,50个域名（存在） 已处理：pm76d1743d49b0464eb8ef2f884dece9e9,49个域名（存在） 已处理：pm0e83d5aac99c4c7aac37a2d4f341ea26,49个域名（存在50） 已处理：pmbfcd43c094a6445eb5813c2b589da41e,50个域名（配管直接回调） ``` ![](../../../../attachment/images-uuid/1c367eebcb444ded91df774e2b698cce.png) 时间 +8 ## 配管确认 ![](../../../../attachment/images-paste/image-20240118131851659.png) ``` pme58e96a1bcc642a8b54c0abcab823e14  1月10日10点回调 {'success': True, 'messageId': 'pme58e96a1bcc642a8b54c0abcab823e14', 'message': 'OK', 'status': 0} pmc32b4c985b3f4816a833c863241e23f4  1月10日15点10分回调 {'success': True, 'messageId': 'pmc32b4c985b3f4816a833c863241e23f4', 'message': 'OK', 'status': 0} pm76d1743d49b0464eb8ef2f884dece9e9  1月10日15点50回调{'success': True, 'messageId': 'pm76d1743d49b0464eb8ef2f884dece9e9', 'message': 'OK', 'status': 0} pm0e83d5aac99c4c7aac37a2d4f341ea26  1月08日17点40回调{'success': True, 'messageId': 'pm0e83d5aac99c4c7aac37a2d4f341ea26', 'message': 'OK', 'status': 0} 2024年1月15日9点51回调 {'success': True, 'messageId': 'pmbfcd43c094a6445eb5813c2b589da41e', 'message': 'OK', 'status': 0} ``` ``` "{""taskId"":""pm8cabcec13bf1415baa720f92ec0d8b3b_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pmbb2bc0bc3d0043b6b99c39afa14c3983_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pm921dd06557ee485f95e866b3a9e2e7d2_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pm28db7a5c4cf8419e8c1be00f19cd8ee1_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pm9d09ca3aee914adeae52023ddb69bbef_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pm0e83d5aac99c4c7aac37a2d4f341ea26_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pme58e96a1bcc642a8b54c0abcab823e14_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pmc32b4c985b3f4816a833c863241e23f4_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pm76d1743d49b0464eb8ef2f884dece9e9_exc"",""type"":""DOMAIN_B_PIPE""}" "{""taskId"":""pmee14b784ec6b463bbf7cc40d5d6105e8_exc"",""type"":""DOMAIN_B_PIPE""}" ``` ## 回调记录 ``` SELECT * FROM sys_task_process_alarm WHERE tag LIKE '%pm0e83d5aac99c4c7aac37a2d4f341ea26%' OR tag LIKE '%pm76d1743d49b0464eb8ef2f884dece9e9%' OR tag LIKE '%pmbfcd43c094a6445eb5813c2b589da41e%' OR tag LIKE '%pmc32b4c985b3f4816a833c863241e23f4%' OR tag LIKE '%pme58e96a1bcc642a8b54c0abcab823e14%'; ``` www.sd-tx.com.cn ![](../../../../attachment/images-paste/image-20240115103320641.png) ## 信安认证规则 1、token 有效期 1 小时 2、立即生效，不需要等 4、会立即失效，同一时间只有一个 token 能用 ## 举例说明 pm0e83d5aac99c4c7aac37a2d4f341ea26   1 月 08 日 17 点 40 回调{'success': True, 'messageId': 'pm0e83d5aac99c4c7aac37a2d4f341ea26', 'message': 'OK', 'status': 0} ``` 2024-01-08 at 17:39:56 CST [http-nio-8080-exec-22007] INFO  com.cmcc.cdn.platform.selfservice.controller.pm.PmPipeCallbackController 47 addBpmContents - 蓝鲸配管 管道配置回调报文，request:{"data":[{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"wjw.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"cgw.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"gzw.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"tyj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"www.whjm.com"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"mzj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"tjj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"jxj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"cgj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"zrzyhghj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"ylj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"swj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"zrzyhgh.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"jtj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"sfj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"sfj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"slhp.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"rfb.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"mzj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"jrj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"jxj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"jtj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"gzw.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"gxs.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"cjw.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"jyj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"mfb.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"gxs.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"czj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"fgj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"nyj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"fgw.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"hbj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"nyncj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"kjj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"wjw.xiaogan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"www.sd-tx.com.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"zwfwzx.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"rsj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"tjj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"va.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"wtxgj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"fgw.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"jyj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"gaj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"rsj.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"home.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"sw.wuhan.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"yjj.ezhou.gov.cn"},{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"scjg.ezhou.gov.cn"}],"messageId":"pm0e83d5aac99c4c7aac37a2d4f341ea26"} ``` ``` 2024-01-08 at 17:39:56 CST [http-nio-8080-exec-22034] INFO  com.cmcc.cdn.platform.selfservice.controller.pm.PmPipeCallbackController 47 addBpmContents - 蓝鲸配管 管道配置回调报文，request:{"data":[{"percent":1.0,"process":false,"reqType":"SELF_DOMAIN_DELETE","targetName":"yjj.wuhan.gov.cn"}],"messageId":"pm8e8c9af3ef854644b412eacb9db15353"} ``` ### 示例 1 ![](../../../../attachment/images-paste/image-20240118110434092.png) ![](../../../../attachment/images-paste/image-20240118110605781.png) #### 对比日志 ``` 2024-01-08 at 17:39:56 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.service.taskprocess.TaskProcessHandleServiceImpl 61 handleEvent2 - start handle type:DOMAIN_B_PIPE, taskid:pm8e8c9af3ef854644b412eacb9db15353_exc 2024-01-08 at 17:39:56 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.rabbitmq.consumer.PipeCallBackDomainRecordConsumer 86 handleMessageId - [管道化任务回调]--->处理回调域名,messageId:pm8e8c9af3ef854644b412eacb9db15353 2024-01-08 at 17:39:56 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.service.impl.BigCloudBasicInfoServiceImpl 1272 getToken - 【请求信安获取认证接口】同步信安，获取token, 参数串:client_id=APrqo8Q8iLF5FslDRR5IfS0zXuADVH4UGYahA4kc&grant_type=password&password=dufLAhOSYFM9&username=Ydcloud2020z 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 342 doHttpRequest2 - url:https://117.136.191.90:5151/oauth/token, method:post, 返回的状态码为:200 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 348 doHttpRequest2 - url:https://117.136.191.90:5151/oauth/token, method:post, 返回的请求体：{"access_token": "x6gvvHFfCe10LqROnGQoJxsxL0lI72", "token_type": "Bearer", "expires_in": 3600, "refresh_token": "NQSd1j0gweUmPlhIgn2QUcDSUdwF51", "scope": ""} 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.service.impl.BigCloudBasicInfoServiceImpl 969 lambda$repeatXinan$0 - 【请求信安接口】同步请求, param:{"delete_user_info_list":[{"user_id_cdn":"bxzDRnTEM"}]} 2024-01-08 at 17:39:58 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 342 doHttpRequest2 - url:https://117.136.191.90:5151/v1.0/delete_user_info, method:post, 返回的状态码为:200 2024-01-08 at 17:39:58 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 348 doHttpRequest2 - url:https://117.136.191.90:5151/v1.0/delete_user_info, method:post, 返回的请求体：{"status": 0, "code": 0, "chk_fail_dict_l": [], "success_num": 1} 2024-01-08 at 17:39:56 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.service.taskprocess.TaskProcessHandleServiceImpl 61 handleEvent2 - start handle type:DOMAIN_B_PIPE, taskid:pm0e83d5aac99c4c7aac37a2d4f341ea26_exc 2024-01-08 at 17:39:56 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.rabbitmq.consumer.PipeCallBackDomainRecordConsumer 86 handleMessageId - [管道化任务回调]--->处理回调域名,messageId:pm0e83d5aac99c4c7aac37a2d4f341ea26 2024-01-08 at 17:39:56 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.service.impl.BigCloudBasicInfoServiceImpl 1272 getToken - 【请求信安获取认证接口】同步信安，获取token, 参数串:client_id=APrqo8Q8iLF5FslDRR5IfS0zXuADVH4UGYahA4kc&grant_type=password&password=dufLAhOSYFM9&username=Ydcloud2020z 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 342 doHttpRequest2 - url:https://117.136.191.90:5151/oauth/token, method:post, 返回的状态码为:200 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 348 doHttpRequest2 - url:https://117.136.191.90:5151/oauth/token, method:post, 返回的请求体：{"access_token": "5oX1XHCvKlBoQ8iHZNvW6D2rw4OKTX", "token_type": "Bearer", "expires_in": 3600, "refresh_token": "5ILveYDkDUGyzl8TwMQm66gPy7U6qt", "scope": ""} 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.log.MonitorJsonLogUtil 110 jsonLogRemote - {"fromServer":"ibs-portal","monitorType":"httpQps","role":"","timeConsuming":642,"method":"https://117.136.191.90:5151/oauth/token","guid":0,"enterpriseId":"","businessType":"monitorTime"} 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.selfservice.service.impl.BigCloudBasicInfoServiceImpl 969 lambda$repeatXinan$0 - 【请求信安接口】同步请求, param:{"delete_user_info_list":[{"user_id_cdn":"yrkHjgZfy"}]} 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 342 doHttpRequest2 - url:https://117.136.191.90:5151/v1.0/delete_user_info, method:post, 返回的状态码为:401 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.HttpUtil 348 doHttpRequest2 - url:https://117.136.191.90:5151/v1.0/delete_user_info, method:post, 返回的请求体：{"_status": "ERR", "_error": {"message": "The server could not verify that you are authorized to access the URL requested. You either supplied the wrong credentials (e.g. a bad password), or your browser doesn't understand how to supply the credentials required.", "code": 401}} 2024-01-08 at 17:39:57 CST [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#15-1] INFO com.cmcc.cdn.platform.common.util.log.MonitorJsonLogUtil 110 jsonLogRemote - {"fromServer":"ibs-portal","monitorType":"httpQps","role":"","timeConsuming":379,"method":"https://117.136.191.90:5151/v1.0/delete_user_info","guid":0,"enterpriseId":"","businessType":"monitorTime"} ``` ### 示例 2 ![](../../../../attachment/images-paste/image-20240118125410377.png) ![](../../../../attachment/images-paste/image-20240118125446428.png) pme6abd2141c624607a825f3f0b64f3ccb 收到客响说未删除成功 pma214b5df90f6458d92fa569f238a8449 执行成功 ![](../../../../attachment/images-paste/image-20240118125712115.png) ![](../../../../attachment/images-paste/image-20240118125825998.png) ![](../../../../attachment/images-paste/image-20240118125651779.png) ![](../../../../attachment/images-paste/image-20240118125851828.png) ## 调用链 com.cmcc.cdn.platform.selfservice.controller.pm.PmPipeCallbackController#addBpmContents com.cmcc.cdn.platform.selfservice.pipeline.service.impl.PipeConfigServiceImpl#dealWithCallBackData com.cmcc.cdn.platform.selfservice.pipeline.service.impl.PipeConfigServiceImpl#pipeWithThread com.cmcc.cdn.platform.selfservice.rabbitmq.consumer.PipeCallBackDomainRecordConsumer#handleMessageId com.cmcc.cdn.platform.selfservice.rabbitmq.consumer.PipeCallBackDomainRecordConsumer#pipeLineCallback com.cmcc.cdn.platform.selfservice.service.impl.DeleteDomainServiceImpl#deleteDomainAndRelatedConfig ![](../../../../attachment/images-paste/image-20240118132627994.png) ## 个人总结 此问题出现在配管方会出现在同一时间回调我们两个任务。这时候如果信安 token 如果是过期的状态，那么会出现两台服务器如 116 和 117 同一秒申请 token。则这时候后申请的会覆盖先申请的，导致先申请的 token 失效。然后一个任务下的域名就不会被删除。