DEV Community

Truman
Truman

Posted on

ApiManager 的分析

WEB MVC的一些基础配置 中,ApiManager类是通过内部的InspectConfiguration类引用的。InspectConfiguration类使用了@Configuration和@ConditionalOnClass注解,表明当Spring上下文中存在RedisTemplate和InterceptorRegistry类时,会自动配置ApiManager。

1. 在InspectConfiguration中引用ApiManager

@Configuration
@ConditionalOnClass({RedisTemplate.class, InterceptorRegistry.class})
@EnableConfigurationProperties(ApiControlProperties.class)
static class InspectConfiguration {

    // 配置ApiManager Bean,负责管理API统计数据
    @Bean
    public ApiManager handlerMethodRedisRegistry(
            RedisTemplate<String, Object> redisTemplate,
            RequestMappingHandlerMapping requestMappingHandlerMapping) {
        return new ApiManager(redisTemplate, requestMappingHandlerMapping);
    }

    // 配置ApiControlInterceptor的自定义拦截器注册器,添加拦截器到注册器
    @Bean
    InterceptorRegistryCustomizer apiControlInterceptorCustomizer(ApiControlProperties apiControlProperties,
                                                                  @Autowired(required = false) CacheControl cacheControl,
                                                                  ApplicationContext applicationContext) {
        return registry -> registry.addInterceptor(new ApiControlInterceptor(apiControlProperties, cacheControl, applicationContext))
                .order(Ordered.HIGHEST_PRECEDENCE + 11) // 设置拦截器的优先级
                .addPathPatterns("/**"); // 设置拦截的路径模式
    }
}

Enter fullscreen mode Exit fullscreen mode

解析

1. @ConditionalOnClass:

  • 这个注解表示InspectConfiguration配置类仅在RedisTemplate和InterceptorRegistry这两个类存在于类路径时才会生效。这意味着只有在应用程序使用Redis并且包含Spring Web MVC时,才会创建ApiManager的Bean。

2. handlerMethodRedisRegistry 方法:

  • 该方法通过@Bean注解声明了一个ApiManager的Bean实例,它通过构造函数接收两个参数:
    • RedisTemplate:用于与Redis进行交互。
    • RequestMappingHandlerMapping:Spring MVC中的映射处理器,用于获取所有Handler方法信息。
  • 在handlerMethodRedisRegistry方法中,ApiManager被创建并注册到Spring容器中,作为一个可注入的Bean。这个ApiManager实例负责处理与API管理相关的逻辑,包括未使用API的管理、请求的统计和推送。

3. apiControlInterceptorCustomizer 方法:

  • 该方法配置了一个自定义的拦截器注册器,返回了一个InterceptorRegistryCustomizer接口的实现。
  • 在这个实现中,ApiControlInterceptor拦截器被添加到了Spring的拦截器链中,优先级非常高(Ordered.HIGHEST_PRECEDENCE + 11),并且该拦截器应用于所有请求路径(/**)。
  • ApiControlInterceptor会利用ApiManager来进行API请求的监控和统计。

ApiManager class

public class ApiManager {

    private final BoundSetOperations<String, Object> unusedApis;

    private final BoundHashOperations<String, String, Number> apiStatisticsHashOps;

    private final RedisTemplate<String, Object> redisTemplate;

    private volatile long lastSetExpireTime;

    public ApiManager(RedisTemplate<String, Object> redisTemplate, RequestMappingHandlerMapping rmhm) {

        this.redisTemplate = redisTemplate;

        String name = Objects.requireNonNull(SpringContextHolder.getRedisPrefixByApplicationName());
        this.unusedApis = redisTemplate.boundSetOps("entropy-reduction:unused-apis:" + name);
        this.apiStatisticsHashOps = redisTemplate.boundHashOps(name + ":api-statistics");

        HashOperations<String, String, String> hashOps = redisTemplate.opsForHash();
        String registerKey = "entropy-reduction:unused-apis:register";
        if (hashOps.get(registerKey, name) == null) {
            /**
             * 熵减工程之去除无用接口
             */
            hashOps.put(registerKey, name, DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss"));

            Object[] handlerMethods = rmhm.getHandlerMethods().values().stream().map(hm -> {
                Method method = hm.getMethod();
                return method.getDeclaringClass().getName() + "." + method.getName();
            }).distinct().filter(item -> item.startsWith("com.example")).toArray();

            unusedApis.add(handlerMethods);
        }
    }

    /**
     * 把请求存起来分析次数和权重
     */
    public void requestToRedis(String deviceId, String host, String clientType) {
        if (StringUtils.isBlank(deviceId)
                || StringUtils.isBlank(clientType)
                || StringUtils.isBlank(host)) {
            return;
        }

        if(DeviceUtils.isApp(clientType)) {
            String formattedDate = DateFormatUtils.format(new Date(), "yyyyMMdd");

            BoundHashOperations<String, String, Number> statisticalWeightHashOps =
                    redisTemplate.boundHashOps("project-common-core:ApiManager:requestToRedis:requestTotal:" + formattedDate);

            statisticalWeightHashOps.increment(clientType + " " + host, 1);

            if (System.currentTimeMillis() - lastSetExpireTime > 3600_000) {
                lastSetExpireTime = System.currentTimeMillis();
                statisticalWeightHashOps.expire(7, TimeUnit.DAYS);
            }

        }
    }

    public void markAsUsed(Set<String> calledApis) {
        unusedApis.remove(calledApis.toArray());
    }


    public Map<String, ApiStatistics> pushAndFetchApiStatistics(Map<String, ApiStatistics> input) {
        for (Map.Entry<String, ApiStatistics> entry : input.entrySet()) {
            this.apiStatisticsHashOps.increment(entry.getKey() + "-duration", entry.getValue().getTotalDuration());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-total", entry.getValue().getTotalCount());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-accept", entry.getValue().getTotalAcceptCount());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-reject", entry.getValue().getTotalRejectCount());
        }

        Map<String, Number> entries = this.apiStatisticsHashOps.entries();

        if (entries != null) {
            Map<String, ApiStatistics> output = new HashMap<>();

            entries.forEach((key, value) -> {
                String[] segments = key.split("-");
                ApiStatistics apiCall = new ApiStatistics();
                switch (segments[1]) {
                    case "duration":
                        apiCall.setTotalDuration(value);
                        break;
                    case "total":
                        apiCall.setTotalCount(value);
                        break;
                    case "accept":
                        apiCall.setTotalAcceptCount(value);
                        break;
                    case "reject":
                        apiCall.setTotalRejectCount(value);
                        break;
                }

                ApiStatistics absent = output.putIfAbsent(segments[0], apiCall);
                if (absent != null) {
                    absent.incrementTotalCount(apiCall.getTotalCount());
                    absent.incrementDuration(apiCall.getTotalDuration());
                    absent.incrementReject(apiCall.getTotalRejectCount());
                    absent.incrementAccept(apiCall.getTotalAcceptCount());
                }
            });
            StringBuilder sb = new StringBuilder();
            output.forEach((key, value) -> {
                sb.append(key).append("\n");
                sb.append("\t-totalCount: ").append(value.getTotalCount()).append("\n");
                sb.append("\t-totalDuration: ").append(value.getTotalDuration()).append("ms\n");
                sb.append("\t-avgDuration: ").append(value.getAvgDuration()).append("ms\n");
                sb.append("\t-totalReject: ").append(value.getTotalRejectCount()).append("\n");
                sb.append("\t-totalAccept: ").append(value.getTotalAcceptCount()).append("\n\n");
                sb.append("-----------------------------------------------------------------\n");
            });
            try {
                FileUtils.writeStringToFile(new File("api-stat.txt"), sb.toString(), "UTF-8");
            } catch (Exception ignore) {
            }
            return output;
        } else {
            return new HashMap<>(input);
        }
    }

}
Enter fullscreen mode Exit fullscreen mode

1. 类成员变量

private final BoundSetOperations<String, Object> unusedApis;
private final BoundHashOperations<String, String, Number> apiStatisticsHashOps;
private final RedisTemplate<String, Object> redisTemplate;
private volatile long lastSetExpireTime;
Enter fullscreen mode Exit fullscreen mode
  • unusedApis: 使用BoundSetOperations接口操作Redis中的一个集合,用于存储未使用的API。
  • apiStatisticsHashOps: 使用BoundHashOperations接口操作Redis中的哈希表,用于存储每个API的统计信息。
  • redisTemplate: 用于与Redis交互的模板,提供操作Redis的各种方法。
  • lastSetExpireTime: 存储上次设置Redis键过期时间的时间戳,用于控制过期时间的设置频率。

2. 构造方法

public ApiManager(RedisTemplate<String, Object> redisTemplate, RequestMappingHandlerMapping rmhm) {
    this.redisTemplate = redisTemplate;

    String name = Objects.requireNonNull(SpringContextHolder.getRedisPrefixByApplicationName());
    this.unusedApis = redisTemplate.boundSetOps("entropy-reduction:unused-apis:" + name);
    this.apiStatisticsHashOps = redisTemplate.boundHashOps(name + ":api-statistics");

    HashOperations<String, String, String> hashOps = redisTemplate.opsForHash();
    String registerKey = "entropy-reduction:unused-apis:register";
    if (hashOps.get(registerKey, name) == null) {
        /**
         * 熵减工程之去除无用接口
         */
        hashOps.put(registerKey, name, DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss"));

        Object[] handlerMethods = rmhm.getHandlerMethods().values().stream().map(hm -> {
            Method method = hm.getMethod();
            return method.getDeclaringClass().getName() + "." + method.getName();
        }).distinct().filter(item -> item.startsWith("com.example")).toArray();

        unusedApis.add(handlerMethods);
    }
}
Enter fullscreen mode Exit fullscreen mode
  • redisTemplate初始化:构造函数接收RedisTemplate和RequestMappingHandlerMapping作为参数,初始化与Redis相关的操作对象unusedApis和apiStatisticsHashOps。
  • 初始化未使用的API集合:
    • 通过RequestMappingHandlerMapping获取所有注册的Handler方法,并将其作为候选未使用的API。
    • 如果Redis中不存在当前应用的未使用API的记录,则将其注册到Redis中,目的是进行API的熵减,即移除不再使用的API。

3. 请求信息存储方法

public void requestToRedis(String deviceId, String host, String clientType) {
    if (StringUtils.isBlank(deviceId) || StringUtils.isBlank(clientType) || StringUtils.isBlank(host)) {
        return;
    }

    if (DeviceUtils.isApp(clientType)) {
        String formattedDate = DateFormatUtils.format(new Date(), "yyyyMMdd");

        BoundHashOperations<String, String, Number> statisticalWeightHashOps =
                redisTemplate.boundHashOps("project-common-core:ApiManager:requestToRedis:requestTotal:" + formattedDate);

        statisticalWeightHashOps.increment(clientType + " " + host, 1);

        if (System.currentTimeMillis() - lastSetExpireTime > 3600_000) {
            lastSetExpireTime = System.currentTimeMillis();
            statisticalWeightHashOps.expire(7, TimeUnit.DAYS);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

功能解析:

  • 请求参数的存储:
    • 方法检查传入的deviceId、host和clientType是否为空,如果为空则直接返回。
    • 如果clientType符合应用客户端的要求,则将请求数据存储到Redis中的哈希结构中,以统计该客户端的请求总数。
  • 过期时间设置:
    • 每隔一小时会设置Redis键的过期时间为7天,以确保统计数据不会无限增长。

4. 标记使用的API

public void markAsUsed(Set<String> calledApis) {
    unusedApis.remove(calledApis.toArray());
}
Enter fullscreen mode Exit fullscreen mode
  • 移除已使用的API:
    • 方法接收一个已调用的API集合calledApis,并将其从unusedApis集合中移除。这一步是为了更新Redis中未使用API的记录,避免将活跃的API误认为未使用。

5. 推送与获取API统计信息

public Map<String, ApiStatistics> pushAndFetchApiStatistics(Map<String, ApiStatistics> input) {
    for (Map.Entry<String, ApiStatistics> entry : input.entrySet()) {
        this.apiStatisticsHashOps.increment(entry.getKey() + "-duration", entry.getValue().getTotalDuration());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-total", entry.getValue().getTotalCount());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-accept", entry.getValue().getTotalAcceptCount());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-reject", entry.getValue().getTotalRejectCount());
    }

    Map<String, Number> entries = this.apiStatisticsHashOps.entries();

    if (entries != null) {
        Map<String, ApiStatistics> output = new HashMap<>();

        entries.forEach((key, value) -> {
            String[] segments = key.split("-");
            ApiStatistics apiCall = new ApiStatistics();
            switch (segments[1]) {
                case "duration":
                    apiCall.setTotalDuration(value);
                    break;
                case "total":
                    apiCall.setTotalCount(value);
                    break;
                case "accept":
                    apiCall.setTotalAcceptCount(value);
                    break;
                case "reject":
                    apiCall.setTotalRejectCount(value);
                    break;
            }

            ApiStatistics absent = output.putIfAbsent(segments[0], apiCall);
            if (absent != null) {
                absent.incrementTotalCount(apiCall.getTotalCount());
                absent.incrementDuration(apiCall.getTotalDuration());
                absent.incrementReject(apiCall.getTotalRejectCount());
                absent.incrementAccept(apiCall.getTotalAcceptCount());
            }
        });

        StringBuilder sb = new StringBuilder();
        output.forEach((key, value) -> {
            sb.append(key).append("\n");
            sb.append("\t-totalCount: ").append(value.getTotalCount()).append("\n");
            sb.append("\t-totalDuration: ").append(value.getTotalDuration()).append("ms\n");
            sb.append("\t-avgDuration: ").append(value.getAvgDuration()).append("ms\n");
            sb.append("\t-totalReject: ").append(value.getTotalRejectCount()).append("\n");
            sb.append("\t-totalAccept: ").append(value.getTotalAcceptCount()).append("\n\n");
            sb.append("-----------------------------------------------------------------\n");
        });
        try {
            FileUtils.writeStringToFile(new File("api-stat.txt"), sb.toString(), "UTF-8");
        } catch (Exception ignore) {
        }
        return output;
    } else {
        return new HashMap<>(input);
    }
}
Enter fullscreen mode Exit fullscreen mode
  • 推送统计数据到Redis:
    • 方法遍历输入的ApiStatistics对象,将每个API的统计数据(包括请求持续时间、总请求数、接受请求数、拒绝请求数)推送到Redis中的哈希结构中。
  • 从Redis获取最新的API统计数据:
    • 从Redis中获取统计数据后,构造一个新的ApiStatistics对象,用于存储从Redis中获取的API统计信息。
  • 合并与写入文件:
    • 将从Redis获取到的统计信息与本地的统计信息合并,然后将统计结果写入到api-stat.txt文件中,用于后续分析。

Image of Datadog

Measure and Advance Your DevSecOps Maturity

In this white paper, we lay out a DevSecOps maturity model based on our experience helping thousands of organizations advance their DevSecOps practices. Learn the key competencies and practices across four distinct levels of maturity.

Get The White Paper

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more