DEV Community

Truman
Truman

Posted on

ApiManager 的分析

在 WEB MVC的一些基础配置 中，ApiManager类是通过内部的InspectConfiguration类引用的。InspectConfiguration类使用了@Configuration和@ConditionalOnClass注解，表明当Spring上下文中存在RedisTemplate和InterceptorRegistry类时，会自动配置ApiManager。

1. 在InspectConfiguration中引用ApiManager

@Configuration
@ConditionalOnClass({RedisTemplate.class, InterceptorRegistry.class})
@EnableConfigurationProperties(ApiControlProperties.class)
static class InspectConfiguration {

    // 配置ApiManager Bean，负责管理API统计数据
    @Bean
    public ApiManager handlerMethodRedisRegistry(
            RedisTemplate<String, Object> redisTemplate,
            RequestMappingHandlerMapping requestMappingHandlerMapping) {
        return new ApiManager(redisTemplate, requestMappingHandlerMapping);
    }

    // 配置ApiControlInterceptor的自定义拦截器注册器，添加拦截器到注册器
    @Bean
    InterceptorRegistryCustomizer apiControlInterceptorCustomizer(ApiControlProperties apiControlProperties,
                                                                  @Autowired(required = false) CacheControl cacheControl,
                                                                  ApplicationContext applicationContext) {
        return registry -> registry.addInterceptor(new ApiControlInterceptor(apiControlProperties, cacheControl, applicationContext))
                .order(Ordered.HIGHEST_PRECEDENCE + 11) // 设置拦截器的优先级
                .addPathPatterns("/**"); // 设置拦截的路径模式
    }
}
Enter fullscreen mode Exit fullscreen mode

解析：

1. @ConditionalOnClass：

  • 这个注解表示InspectConfiguration配置类仅在RedisTemplate和InterceptorRegistry这两个类存在于类路径时才会生效。这意味着只有在应用程序使用Redis并且包含Spring Web MVC时，才会创建ApiManager的Bean。

2. handlerMethodRedisRegistry 方法：

  • 该方法通过@Bean注解声明了一个ApiManager的Bean实例，它通过构造函数接收两个参数：
    • RedisTemplate：用于与Redis进行交互。
    • RequestMappingHandlerMapping：Spring MVC中的映射处理器，用于获取所有Handler方法信息。
  • 在handlerMethodRedisRegistry方法中，ApiManager被创建并注册到Spring容器中，作为一个可注入的Bean。这个ApiManager实例负责处理与API管理相关的逻辑，包括未使用API的管理、请求的统计和推送。

3. apiControlInterceptorCustomizer 方法：

  • 该方法配置了一个自定义的拦截器注册器，返回了一个InterceptorRegistryCustomizer接口的实现。
  • 在这个实现中，ApiControlInterceptor拦截器被添加到了Spring的拦截器链中，优先级非常高（Ordered.HIGHEST_PRECEDENCE + 11），并且该拦截器应用于所有请求路径（/**）。
  • ApiControlInterceptor会利用ApiManager来进行API请求的监控和统计。

ApiManager class

public class ApiManager {

    private final BoundSetOperations<String, Object> unusedApis;

    private final BoundHashOperations<String, String, Number> apiStatisticsHashOps;

    private final RedisTemplate<String, Object> redisTemplate;

    private volatile long lastSetExpireTime;

    public ApiManager(RedisTemplate<String, Object> redisTemplate, RequestMappingHandlerMapping rmhm) {

        this.redisTemplate = redisTemplate;

        String name = Objects.requireNonNull(SpringContextHolder.getRedisPrefixByApplicationName());
        this.unusedApis = redisTemplate.boundSetOps("entropy-reduction:unused-apis:" + name);
        this.apiStatisticsHashOps = redisTemplate.boundHashOps(name + ":api-statistics");

        HashOperations<String, String, String> hashOps = redisTemplate.opsForHash();
        String registerKey = "entropy-reduction:unused-apis:register";
        if (hashOps.get(registerKey, name) == null) {
            /**
             * 熵减工程之去除无用接口
             */
            hashOps.put(registerKey, name, DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss"));

            Object[] handlerMethods = rmhm.getHandlerMethods().values().stream().map(hm -> {
                Method method = hm.getMethod();
                return method.getDeclaringClass().getName() + "." + method.getName();
            }).distinct().filter(item -> item.startsWith("com.example")).toArray();

            unusedApis.add(handlerMethods);
        }
    }

    /**
     * 把请求存起来分析次数和权重
     */
    public void requestToRedis(String deviceId, String host, String clientType) {
        if (StringUtils.isBlank(deviceId)
                || StringUtils.isBlank(clientType)
                || StringUtils.isBlank(host)) {
            return;
        }

        if(DeviceUtils.isApp(clientType)) {
            String formattedDate = DateFormatUtils.format(new Date(), "yyyyMMdd");

            BoundHashOperations<String, String, Number> statisticalWeightHashOps =
                    redisTemplate.boundHashOps("project-common-core:ApiManager:requestToRedis:requestTotal:" + formattedDate);

            statisticalWeightHashOps.increment(clientType + " " + host, 1);

            if (System.currentTimeMillis() - lastSetExpireTime > 3600_000) {
                lastSetExpireTime = System.currentTimeMillis();
                statisticalWeightHashOps.expire(7, TimeUnit.DAYS);
            }

        }
    }

    public void markAsUsed(Set<String> calledApis) {
        unusedApis.remove(calledApis.toArray());
    }


    public Map<String, ApiStatistics> pushAndFetchApiStatistics(Map<String, ApiStatistics> input) {
        for (Map.Entry<String, ApiStatistics> entry : input.entrySet()) {
            this.apiStatisticsHashOps.increment(entry.getKey() + "-duration", entry.getValue().getTotalDuration());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-total", entry.getValue().getTotalCount());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-accept", entry.getValue().getTotalAcceptCount());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-reject", entry.getValue().getTotalRejectCount());
        }

        Map<String, Number> entries = this.apiStatisticsHashOps.entries();

        if (entries != null) {
            Map<String, ApiStatistics> output = new HashMap<>();

            entries.forEach((key, value) -> {
                String[] segments = key.split("-");
                ApiStatistics apiCall = new ApiStatistics();
                switch (segments[1]) {
                    case "duration":
                        apiCall.setTotalDuration(value);
                        break;
                    case "total":
                        apiCall.setTotalCount(value);
                        break;
                    case "accept":
                        apiCall.setTotalAcceptCount(value);
                        break;
                    case "reject":
                        apiCall.setTotalRejectCount(value);
                        break;
                }

                ApiStatistics absent = output.putIfAbsent(segments[0], apiCall);
                if (absent != null) {
                    absent.incrementTotalCount(apiCall.getTotalCount());
                    absent.incrementDuration(apiCall.getTotalDuration());
                    absent.incrementReject(apiCall.getTotalRejectCount());
                    absent.incrementAccept(apiCall.getTotalAcceptCount());
                }
            });
            StringBuilder sb = new StringBuilder();
            output.forEach((key, value) -> {
                sb.append(key).append("\n");
                sb.append("\t-totalCount: ").append(value.getTotalCount()).append("\n");
                sb.append("\t-totalDuration: ").append(value.getTotalDuration()).append("ms\n");
                sb.append("\t-avgDuration: ").append(value.getAvgDuration()).append("ms\n");
                sb.append("\t-totalReject: ").append(value.getTotalRejectCount()).append("\n");
                sb.append("\t-totalAccept: ").append(value.getTotalAcceptCount()).append("\n\n");
                sb.append("-----------------------------------------------------------------\n");
            });
            try {
                FileUtils.writeStringToFile(new File("api-stat.txt"), sb.toString(), "UTF-8");
            } catch (Exception ignore) {
            }
            return output;
        } else {
            return new HashMap<>(input);
        }
    }

}
Enter fullscreen mode Exit fullscreen mode

1. 类成员变量

private final BoundSetOperations<String, Object> unusedApis;
private final BoundHashOperations<String, String, Number> apiStatisticsHashOps;
private final RedisTemplate<String, Object> redisTemplate;
private volatile long lastSetExpireTime;
Enter fullscreen mode Exit fullscreen mode
  • unusedApis: 使用BoundSetOperations接口操作Redis中的一个集合，用于存储未使用的API。
  • apiStatisticsHashOps: 使用BoundHashOperations接口操作Redis中的哈希表，用于存储每个API的统计信息。
  • redisTemplate: 用于与Redis交互的模板，提供操作Redis的各种方法。
  • lastSetExpireTime: 存储上次设置Redis键过期时间的时间戳，用于控制过期时间的设置频率。

2. 构造方法

public ApiManager(RedisTemplate<String, Object> redisTemplate, RequestMappingHandlerMapping rmhm) {
    this.redisTemplate = redisTemplate;

    String name = Objects.requireNonNull(SpringContextHolder.getRedisPrefixByApplicationName());
    this.unusedApis = redisTemplate.boundSetOps("entropy-reduction:unused-apis:" + name);
    this.apiStatisticsHashOps = redisTemplate.boundHashOps(name + ":api-statistics");

    HashOperations<String, String, String> hashOps = redisTemplate.opsForHash();
    String registerKey = "entropy-reduction:unused-apis:register";
    if (hashOps.get(registerKey, name) == null) {
        /**
         * 熵减工程之去除无用接口
         */
        hashOps.put(registerKey, name, DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss"));

        Object[] handlerMethods = rmhm.getHandlerMethods().values().stream().map(hm -> {
            Method method = hm.getMethod();
            return method.getDeclaringClass().getName() + "." + method.getName();
        }).distinct().filter(item -> item.startsWith("com.example")).toArray();

        unusedApis.add(handlerMethods);
    }
}
Enter fullscreen mode Exit fullscreen mode
  • redisTemplate初始化：构造函数接收RedisTemplate和RequestMappingHandlerMapping作为参数，初始化与Redis相关的操作对象unusedApis和apiStatisticsHashOps。
  • 初始化未使用的API集合：
    • 通过RequestMappingHandlerMapping获取所有注册的Handler方法，并将其作为候选未使用的API。
    • 如果Redis中不存在当前应用的未使用API的记录，则将其注册到Redis中，目的是进行API的熵减，即移除不再使用的API。

3. 请求信息存储方法

public void requestToRedis(String deviceId, String host, String clientType) {
    if (StringUtils.isBlank(deviceId) || StringUtils.isBlank(clientType) || StringUtils.isBlank(host)) {
        return;
    }

    if (DeviceUtils.isApp(clientType)) {
        String formattedDate = DateFormatUtils.format(new Date(), "yyyyMMdd");

        BoundHashOperations<String, String, Number> statisticalWeightHashOps =
                redisTemplate.boundHashOps("project-common-core:ApiManager:requestToRedis:requestTotal:" + formattedDate);

        statisticalWeightHashOps.increment(clientType + " " + host, 1);

        if (System.currentTimeMillis() - lastSetExpireTime > 3600_000) {
            lastSetExpireTime = System.currentTimeMillis();
            statisticalWeightHashOps.expire(7, TimeUnit.DAYS);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

功能解析：

  • 请求参数的存储：
    • 方法检查传入的deviceId、host和clientType是否为空，如果为空则直接返回。
    • 如果clientType符合应用客户端的要求，则将请求数据存储到Redis中的哈希结构中，以统计该客户端的请求总数。
  • 过期时间设置：
    • 每隔一小时会设置Redis键的过期时间为7天，以确保统计数据不会无限增长。

4. 标记使用的API

public void markAsUsed(Set<String> calledApis) {
    unusedApis.remove(calledApis.toArray());
}
Enter fullscreen mode Exit fullscreen mode
  • 移除已使用的API：
    • 方法接收一个已调用的API集合calledApis，并将其从unusedApis集合中移除。这一步是为了更新Redis中未使用API的记录，避免将活跃的API误认为未使用。

5. 推送与获取API统计信息

public Map<String, ApiStatistics> pushAndFetchApiStatistics(Map<String, ApiStatistics> input) {
    for (Map.Entry<String, ApiStatistics> entry : input.entrySet()) {
        this.apiStatisticsHashOps.increment(entry.getKey() + "-duration", entry.getValue().getTotalDuration());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-total", entry.getValue().getTotalCount());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-accept", entry.getValue().getTotalAcceptCount());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-reject", entry.getValue().getTotalRejectCount());
    }

    Map<String, Number> entries = this.apiStatisticsHashOps.entries();

    if (entries != null) {
        Map<String, ApiStatistics> output = new HashMap<>();

        entries.forEach((key, value) -> {
            String[] segments = key.split("-");
            ApiStatistics apiCall = new ApiStatistics();
            switch (segments[1]) {
                case "duration":
                    apiCall.setTotalDuration(value);
                    break;
                case "total":
                    apiCall.setTotalCount(value);
                    break;
                case "accept":
                    apiCall.setTotalAcceptCount(value);
                    break;
                case "reject":
                    apiCall.setTotalRejectCount(value);
                    break;
            }

            ApiStatistics absent = output.putIfAbsent(segments[0], apiCall);
            if (absent != null) {
                absent.incrementTotalCount(apiCall.getTotalCount());
                absent.incrementDuration(apiCall.getTotalDuration());
                absent.incrementReject(apiCall.getTotalRejectCount());
                absent.incrementAccept(apiCall.getTotalAcceptCount());
            }
        });

        StringBuilder sb = new StringBuilder();
        output.forEach((key, value) -> {
            sb.append(key).append("\n");
            sb.append("\t-totalCount: ").append(value.getTotalCount()).append("\n");
            sb.append("\t-totalDuration: ").append(value.getTotalDuration()).append("ms\n");
            sb.append("\t-avgDuration: ").append(value.getAvgDuration()).append("ms\n");
            sb.append("\t-totalReject: ").append(value.getTotalRejectCount()).append("\n");
            sb.append("\t-totalAccept: ").append(value.getTotalAcceptCount()).append("\n\n");
            sb.append("-----------------------------------------------------------------\n");
        });
        try {
            FileUtils.writeStringToFile(new File("api-stat.txt"), sb.toString(), "UTF-8");
        } catch (Exception ignore) {
        }
        return output;
    } else {
        return new HashMap<>(input);
    }
}
Enter fullscreen mode Exit fullscreen mode
  • 推送统计数据到Redis：
    • 方法遍历输入的ApiStatistics对象，将每个API的统计数据（包括请求持续时间、总请求数、接受请求数、拒绝请求数）推送到Redis中的哈希结构中。
  • 从Redis获取最新的API统计数据：
    • 从Redis中获取统计数据后，构造一个新的ApiStatistics对象，用于存储从Redis中获取的API统计信息。
  • 合并与写入文件：
    • 将从Redis获取到的统计信息与本地的统计信息合并，然后将统计结果写入到api-stat.txt文件中，用于后续分析。

Top comments (0)

Read next

pmbanugo profile image

We Ditched Kubernetes! Here's Why & What We're Using Now

Peter Mbanugo -

niharikaa profile image

10 Advanced TypeScript Concepts Every Developer Should Know

Niharika Goulikar -

codepo8 profile image

httpstat.us is a great service to test your APIs and scripts

Christian Heilmann -

jess profile image

What was your win this week?

Jess Lee -