DEV Community: Lara Lee

Alibaba Cloud’s Qwen2.5-Max Secures Top Rankings in Chatbot Arena

Lara Lee — Thu, 06 Feb 2025 08:55:45 +0000

Alibaba Cloud’s latest proprietary large language model(LLM), Qwen2.5-Max, has achieved impressive results on Chatbot Arena, a well-recognized open platform that evaluates the world’s best LLM and AI chatbots. Ranked #7 overall in the Arena score, Qwen2.5-Max matches other top proprietary LLMs and demonstrates exceptional capabilities, particularly in technical domains. It ranks #1 position in math and coding and ranks #2 in hard prompts, which involve complex prompts in addressing challenging tasks, solidifying its status as a powerhouse in tackling complex tasks.

As a cutting-edge Mixture of Experts (MoE) model, Qwen2.5-Max has been trained on over 20 trillion tokens and further refined with Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) techniques. Leveraging these technological advancements, Qwen2.5-Max has demonstrated exceptional strengths in knowledge, coding, general capabilities, and human alignment, securing leading scores in major benchmarks including MMLU-Pro, LiveCodeBench, LiveBench, and Arena-Hard.

Click here to read all. Originally published at https://www.alibabacloud.com.

https://discord.com/invite/KPmq628K63

Quit the IT Headaches: Experience a Seamless Migration to Alibaba Cloud Today!

Lara Lee — Wed, 06 Nov 2024 07:04:10 +0000

Migrating to the cloud can feel daunting, especially when juggling everyday business operations. But with the right partner, the journey becomes smoother and more secure than ever. For small and medium-sized businesses (SMBs) looking to enhance operational agility, security, and efficiency, Alibaba Cloud offers the perfect platform for a smooth, low-risk migration. Here’s everything you need to know to confidently make the transition and unlock the true potential of cloud computing.

Why Choose Alibaba Cloud for Your Migration?

Choosing the right cloud provider is essential to ensure that your migration journey is seamless and that you’re well-supported every step of the way. Here’s why Alibaba Cloud is an ideal choice for businesses of any size:

Global Reach with Localized Service: With data centers across 20+ regions globally, including localized support for compliance, Alibaba Cloud is designed for businesses aiming to expand their reach while maintaining speed, data security, and low latency. This is especially beneficial for businesses with regional data residency requirements.
Robust Security and Compliance: Data protection is a top priority for Alibaba Cloud, which provides multi-layered security features like DDoS protection, data encryption, and Identity Access Management (IAM). Compliance with global standards such as GDPR and ISO 27001 ensures businesses meet stringent security and privacy regulations.
Cost-Effective and Scalable Solutions: Alibaba Cloud’s flexible billing structures — including pay-as-you-go and subscription models — allow businesses to adjust their IT expenses based on actual usage. This prevents overspending and provides reliable cost management that scales with your business.
Cutting-Edge Innovation: Beyond storage and computing power, Alibaba Cloud supports advanced technology like AI, big data, and IoT, enabling businesses to stay competitive and innovative with the latest tech.

Key Steps for a Smooth Migration to Alibaba Cloud

A seamless migration to Alibaba Cloud involves several key stages, each designed to minimize risk and maximize efficiency. Let’s break down the steps:

1. Conduct a Comprehensive Pre-Migration Assessment
Before making the transition, analyze your current IT infrastructure, data, and application requirements to determine what should migrate to the cloud. Alibaba Cloud’s migration experts are available to conduct assessments, helping you design an optimal migration strategy tailored to your specific needs.

2. Leverage Alibaba Cloud’s Migration Tools and Services
Alibaba Cloud provides a suite of migration tools to streamline data and application transfers:

Data Transfer Solutions: For large-scale data migration, Alibaba Cloud’s secure tools simplify data transfers with minimal disruptions.
Database Migration Service (DMS): Alibaba Cloud’s DMS supports seamless transfers of databases from on-premises or other cloud environments, ensuring that your data is readily available.
Apsara Stack: For businesses requiring a hybrid cloud setup, Apsara Stack enables efficient management and migration between on-premises and cloud environments.

3. Execute a Phased Migration Approach
To reduce potential risks, consider adopting a phased migration strategy, beginning with less critical workloads or applications. This way, you can test the performance in Alibaba Cloud and ensure uninterrupted service for core operations.

4. Validate and Optimize Post-Migration Performance
After the migration, Alibaba Cloud’s real-time monitoring and management tools help you evaluate system performance and make necessary adjustments to ensure everything runs optimally. With Alibaba Cloud’s support, businesses can further optimize resource usage to maximize performance and reduce costs.

Ensuring Data Security During Migration

Data security is a critical consideration for any migration. Alibaba Cloud provides robust, multi-layered security to protect your data:

End-to-End Data Encryption: With SSL/TLS protocols, data transfers are securely encrypted, ensuring data integrity and confidentiality during the migration process.
Granular Identity Access Management (IAM): IAM allows businesses to manage access permissions effectively, preventing unauthorized data access.
Global Compliance Standards: Alibaba Cloud complies with international data protection standards, providing support for GDPR, ISO, and other industry requirements to ensure businesses meet necessary compliance needs.

Success Stories: How Alibaba Cloud Empowers Smooth Migrations

PT. Catur Sukses Internasional (PT. CSI)
PT. Catur Sukses Internasional (PT. CSI), a prominent Indonesian home appliance distributor, turned to Alibaba Cloud to enhance IT efficiency and data security. With two data centers in Jakarta, Alibaba Cloud was able to meet PT. CSI’s data residency requirements while minimizing network latency. The Alibaba Cloud team worked closely with PT. CSI to design and implement a full migration and operations strategy, helping them shift internal resources from routine maintenance to more strategic business activities. This shift allowed PT. CSI focuses on innovation and growth, while Alibaba Cloud handled the day-to-day infrastructure needs.

enish Co. Ltd.
Based in Tokyo, enish Co. Ltd. is a leading game production company that needed a cloud partner with robust security, scalability, and advanced migration capabilities. Alibaba Cloud provided enish with all the required services, from automated data migration to comprehensive security features, enabling an efficient transfer from their existing on-premises systems. With Alibaba Cloud’s support, enish could streamline their migration and significantly reduce their maintenance hours, freeing up time to focus on game development and user experience improvements.

Conclusion

Migrating to Alibaba Cloud can be a transformative experience for SMBs, offering the flexibility, security, and efficiency that are essential in today’s digital landscape. With comprehensive technical support, a global infrastructure network, and rich migration tools, Alibaba Cloud ensures that businesses enjoy a seamless migration journey with minimal disruption. Make the move today and start enjoying the benefits of a future-ready, secure, and agile cloud infrastructure!

https://discord.gg/KPmq628K63

What You Need to Know About Empowering Your SMB with Cloud Workspaces!

Lara Lee — Tue, 29 Oct 2024 08:45:18 +0000

In today’s rapidly evolving business environment, small and medium-sized businesses (SMBs) face the dual challenge of effectively managing remote work while ensuring data security. According to surveys, over 70% of SMBs believe that remote work increases data security risks. So, how can you enhance team productivity while ensuring security? Alibaba Cloud’s Workspace on Cloud solution is exactly what you need.

How Cloud Workspaces Enhance Business Efficiency and Security

First and foremost, cloud workspaces significantly enhance data security. Compared to traditional on-premises servers, Alibaba Cloud’s Elastic Desktop Service (EDS) utilizes advanced security technologies to ensure sensitive data is protected during transmission and storage. Through centralized management and automated backups, businesses can effectively prevent data loss and breaches, thereby increasing customer trust.

Secondly, cloud workspaces greatly improve team collaboration efficiency. With remote work becoming the norm, team members are often spread across different locations, but Alibaba Cloud’s solutions ensure they can collaborate seamlessly in a secure environment. For example, ONG&ONG Group successfully implemented EDS, configuring over 100 high-performance BIM virtual desktops, which significantly boosted productivity. This case demonstrates that cloud workspaces allow teams to remain efficient while enabling true global collaboration.

Breaking Free from Traditional Constraints: The Flexibility and Cost Advantages of Cloud Solutions

Cloud workspaces not only provide clear advantages in security and efficiency but also greatly surpass traditional methods in terms of cost control and flexibility. By utilizing Alibaba Cloud’s cloud desktop solutions, businesses can reduce hardware investments and eliminate the need for complex on-premises infrastructure. According to market research, companies adopting cloud solutions can reduce their IT costs by an average of 30%. This is particularly important for SMBs with limited budgets.

Furthermore, cloud workspaces offer flexible resource allocation capabilities. Businesses can quickly scale resources up or down according to actual needs, avoiding waste associated with fixed hardware investments. This flexibility enables SMBs to adapt swiftly to market changes and maintain competitiveness.

Additionally, the education and training sector also benefits from the flexibility of cloud workspaces. Alibaba Cloud’s EDS provides educational institutions with personalized learning spaces and interactive teaching tools, allowing students to access learning resources anytime, anywhere. This flexible learning approach not only enhances learning outcomes but also saves educational institutions significant operational costs.

Conclusion

By adopting Alibaba Cloud’s Workspace on Cloud, SMBs can enhance data security and team efficiency, gaining a distinct competitive edge in the market. If you’re still hesitant, it’s advisable to contact Alibaba Cloud’s professional team immediately to discuss how to integrate these solutions into your business. Consider setting up a trial period to test the changes brought by cloud workspaces, such as productivity improvements and reduced operational costs.

As exemplified by ONG&ONG Group’s success, configuring cloud desktops allows you to provide a more flexible work environment for your team, enabling employees to collaborate efficiently from anywhere. Take action now to empower your business with cloud computing, ensuring your SMB progresses steadily in the future market!

Empowering SMBs: The Impact of Data and Analytics Tools in Cloud Computing

Lara Lee — Fri, 25 Oct 2024 09:28:02 +0000

In the current landscape where data drives business decisions, small and medium-sized businesses (SMBs) must effectively harness advanced data and analytics tools to thrive. Cloud computing has fundamentally changed how organizations manage their data, providing robust solutions that enhance efficiency, reduce costs, and facilitate informed decision-making. This blog will explore how Alibaba Cloud’s data and analytics offerings can help your organization boost operational capabilities and support growth.

The Advantages of Cloud-Based Data and Analytics Tools for SMBs

Cost Efficiency:
Traditional data and analytics solutions often demand significant infrastructure investments. Cloud-based tools like Alibaba Cloud’s Quick BI and MaxCompute operate on a pay-as-you-go model, enabling SMBs to minimize capital expenditures. This allows for reallocating resources toward innovation and expansion rather than overhead.
Scalability and Flexibility:
As your business evolves, so do your data requirements. Alibaba Cloud’s DataWorks and MaxCompute provide the flexibility to scale your data and analytics capabilities without the burden of managing physical infrastructure. This adaptability ensures your business can pivot quickly to meet market demands, enhancing your competitive position.
Real-Time Insights for Informed Decisions:
Access to real-time data is crucial for timely decision-making. Cloud solutions enable faster data and analytics processing and reporting, allowing your teams to generate insights quickly. Tools like Quick BI empower your team to create dashboards and visualizations in minutes, keeping everyone aligned and focused on strategic objectives.
Automation and Reduced Workload:
Automating data management tasks can free up your IT team’s resources for more strategic initiatives. Solutions like Sensors Data and Alibaba Cloud’s data transfer services (DTS) streamline data backup and synchronization operations. This reduces manual intervention, allowing your technical team to focus on innovation rather than routine maintenance.

Case Study: Indofun’s Success with Alibaba Cloud

To illustrate the tangible benefits of these technologies, let’s examine Indofun, one of Indonesia’s leading game providers. By leveraging Alibaba Cloud’s computing, storage, and database services, Indofun has achieved remarkable results that can inspire other SMBs.

Reliable Performance for Customer Satisfaction:
Utilizing Alibaba Cloud’s Elastic Compute Service (ECS) and Server Load Balancer (SLB), Indofun ensures minimal latency and fast loading times. This enhances user satisfaction and engagement, demonstrating how reliable performance can drive customer loyalty.
High-Quality User Experience:
Integrating cloud services ensures that Indofun maintains exceptional image and audio quality in real-time, delivering an immersive gaming environment. This high-quality experience is crucial in today’s competitive market, where user expectations are constantly rising.
Comprehensive Service Package:
Alibaba Cloud offers a tailored service package that meets Indofun’s diverse needs. Services like ApsaraDB for Redis and ApsaraDB RDS for MySQL automate routine data tasks, significantly reducing the manual effort required by Indofun’s technical support team. This not only simplifies operations but also enhances overall efficiency.
Cost-Effective Growth Solutions:
By utilizing Alibaba Cloud’s cost-effective solutions, Indofun not only meets customer expectations but also positions itself as a leader in the gaming industry. This showcases how smart investment in data and analytics can lead to substantial returns.

Conclusion

For SMBs aiming for growth and innovation, adopting cloud-based data and analytics tools from Alibaba Cloud can be transformative. These technologies empower businesses to make informed decisions, optimize operations, and respond swiftly to market changes. As the digital landscape continues to evolve, investing in cloud computing and data and analytics is essential for sustained success.

By understanding and leveraging these powerful tools, SMB leaders and tech talents can drive their organizations forward. If you’re ready to elevate your data and analytics capabilities, consider exploring Alibaba Cloud’s suite of analytics solutions designed to support your journey toward growth and excellence.

Enhancing Customer Engagement: Connecting with End Users Through Alibaba Cloud Tools

Lara Lee — Tue, 22 Oct 2024 08:26:43 +0000

In a world overflowing with information, effective communication between businesses and customers is essential for driving customer engagement, resembling the challenge of finding a gem in a bustling market. With the right tools, not only can you discover that gem, but you can also cultivate a lasting relationship with it. Alibaba Cloud provides a suite of powerful tools that empower businesses to connect with customers across various channels effectively.

From CRM to SMS: Elevating Customer Experience

Imagine managing customer data, sending emails, posting on social media, and interacting through messaging apps — all on a single platform! Salesforce on Alibaba Cloud is just that, allowing you to navigate customer management with ease. For instance, when the food giant Tao Kae Noi ventured into the Chinese market, they faced complex communication challenges. By leveraging Alibaba Cloud’s Salesforce, they quickly built customer profiles and analyzed needs in real-time, smoothly launching new products.

Additionally, Alibaba Cloud’s Short Message Service (SMS) and Direct Mail Services streamline communication remarkably. Businesses can swiftly send important notifications to keep in close contact with clients. For example, Tao Kae Noi utilized Alibaba Cloud’s Direct Mail Service to ensure timely communication with customers across China. This way, whether it’s promotional information or product updates, customers are always in the loop.

Breaking Geographic Barriers with Innovation

During the pandemic, many businesses faced unprecedented challenges. This is where Alibaba Cloud’s Chat Application Message Service proved vital. Consider a small startup like SyZyGy, which focuses on virtual experiences and gamified solutions. They launched the Clairvo project to assist corporate clients in hosting online events. With teams spread across Hong Kong and Mainland China, smooth communication was essential. Using Alibaba Cloud’s chat application service, they could interact with clients in real-time on WhatsApp and enhance customer service efficiency with built-in chatbots. Customers received immediate responses, enjoying a seamless service experience.

Partnering with Alibaba Cloud for Success

These businesses found their footing in a competitive market through collaboration with Alibaba Cloud. HungryPanda, a food delivery platform based in London, achieved a staggering 600% increase in demand during the pandemic thanks to its strategic partnership with Alibaba Cloud. Facing such enormous demand, HungryPanda relied on Alibaba Cloud’s cloud-native solutions to ensure efficient and stable service, allowing customers to enjoy authentic Chinese cuisine from the comfort of their homes.

Conclusion

With Alibaba Cloud’s multi-channel communication tools, businesses can efficiently reach customers and optimize engagement strategies through data analysis. Whether through CRM, email, SMS, or social media, Alibaba Cloud’s tools help you secure your place in customers’ hearts. Choosing the right tools is key to realizing customer value. Partner with Alibaba Cloud and let’s search for that shining gem together in a bustling market!

Core Technology of PolarDB-X Storage Engine | Lizard Lock-free Backup

Lara Lee — Mon, 21 Oct 2024 06:54:51 +0000

Backup Background

PolarDB-X distributed database adopts an integrated centralized-distributed architecture. As business development and expansion progress, the Data Node can be gradually expanded from centralized to distributed linearly, smoothly and horizontally split online.

Database backup and restoration, bear an important role in ensuring data security. Based on data protection and disaster recovery requirements, PolarDB-X provides various capabilities to support point-in-time recovery (PITR). For more technical details, please refer to Safeguarding Your Data with PolarDB-X: Backup and Restoration.

As shown in the preceding figure, PolarDB-X is restored in PITR. The combination of the full backup and incremental binlog can meet the recovery requirements at any time.
In addition, as a multi-replica architecture, the data node (DN) of PolarDB-X will trigger dynamic migration and reconstruction of replicas during daily host operation and maintenance. Then it is necessary to rely on the secondary database reconstruction technology. First, a full backup is performed through the primary database and the multi-replica Consensus Log is used to incrementally synchronize the replica data.

This article focuses on the design of full backup as a data node responsible for online data storage. Based on the requirements of data protection and disaster recovery, DN needs to support flexible and customized backup policies. At the same time, PolarDB-X can process and store huge amounts of data and back up these data volumes (such as data sets up to 20TB in size). If you want to complete the backup in a shorter time (maybe 3 to 4 hours), such backup SLAs need to be optimized in terms of online business impact and parallelism strategies.

Backup Challenges of Traditional MySQL

Open-source MySQL uses the InnoDB storage engine to store data. Based on the traditional WAL architecture, it uses redo log physical logs to record data changes. The Server layer uses binlogs as logical logs to record the change logs of database instances, including all database changes such as transactional data modifications, non-transactional data modifications, and schema modifications. It plays three important roles:

1. Two-phase Commit Coordination Logs

In the MySQL structure with multiple storage engines, binlogs assume the responsibility of the coordinator of 2PC and participate in the two-phase commit and recovery process.

2. Data Change Logs

Binlogs record all changes in database instances. You can use the change log to replay a logical image to build a cluster of multiple types.

3. Executed Set Logs

Binlogs record the unique identifier of the change, Global Transaction Identifier, or GTID, to construct the Executed set.
How to ensure the backup consistency between the binlog and InnoDB storage engine poses a challenge. To maintain the consistency between binlogs and storage engine participants, a logical point in time is required, while generally, a global lock is required to implement a logical point in time, which seriously affects the running of online instances and then user business.

Lizard Lock-free Full Backup Design

PolarDB-X storage engine developed the Lizard distributed transaction system to replace the traditional InnoDB standalone transaction system. Given the disadvantages of the InnoDB transaction system, the Lizard transaction system designs the SCN standalone transaction system and the GCN distributed transaction system to solve these disadvantages, effectively supporting distributed database capabilities.

At the same time, compared with the limitations of traditional MySQL physical backup, PolarDB-X introduces the newly designed Lizard lock-free backup based on the Lizard transaction engine, making the backup process transparent to user business and truly imperceptible.

The core of Lizard lock-free backup is to abandon binlog backup during the physical backup process and only retain the backup of storage engine participants. The three most important responsibilities of binlogs are:

1. Two-phase Commit Coordination Logs

All the storage engine participants in the one-phase prepare state are processed based on rollback.

2. Change Logs

When a DN uses a backup set to restore a new instance, it is forced to join the cluster as a follower or learner to receive binlogs sent by the cluster leader using the X-Paxos protocol.

3. Executed Set

PolarDB-X DN introduces a new component, Lizard Transaction Slot (LTS), as a new mandatory participant engine to participate in all MySQL changes, including transactional, non-transactional, and structural changes. The Gtid Executed set is maintained by LTS to achieve idempotent control.

The preceding methods replace the responsibilities of binlogs and achieve Lizard lock-free full backup capability.

Lizard Lock-free Full Backup Architecture

Lizard lock-free full backup introduces a new important component, Transaction Slot, which serves as a persistent structure to record database changes. Its architecture diagram is as follows:

Transaction Slot Component

As a persistent structure, Transaction Slot records the transactional changes of the database, including:

1. Transaction State
The state field is used to record the states of transactions, including active, prepare, commit, and rollback. During the recovery process, the state field is used to perform rollback for all transactions in the active or prepared state.

2. Transaction Commit Version

The SCN field records the internal version number of the transaction commit or rollback, that is, the System Commit Number.

3. Global Transaction Identifier
The GTID field records the unique identifier of the transaction commit or rollback, that is, the Global Transaction Identifier. To ensure record integrity, all changes to the database include:

Data Manipulation Language (DML)
Data Definition Language (DDL)
Data Control Language (DCL)

Changes are all forcibly allocated Transaction Slots to record all GTIDs, thereby ensuring the idempotence of binlogs after the instance is restored.

Lizard Consistency Recovery

Through the Transaction Slot component, PolarDB-X DN no longer uses the global lock to obtain the checkpoint, achieving lock-free full backup. However, during the recovery process for the full backup set, the lock-free full backup loosely processes the three core checkpoints that need to be processed, namely, the Recovery checkpoint, the Consensus checkpoint, and the Apply checkpoint, guaranteeing the final consistency.

1. Recovery Checkpoint

The physical recovery checkpoint takes the last log location copied by the redo log in the physical backup as the final recovery location. All redo logs after the checkpoint location are copied during the backup process and reapplied during recovery, and all transactions in the active/prepare state are rolled back.

2. Consensus Checkpoint

Since DN uses the X-Paxos consistency protocol, whose core is to achieve a majority commit during the commit process, a majority binlog checkpoint is obtained as the Consensus checkpoint before the backup ends. During the instance recovery process, it rejoins the cluster as a Learner and receives binlogs from this checkpoint.

3. Apply Checkpoint

Without the protection of Global Lock, it is impossible to obtain the applied checkpoint with consistency between binlogs and the storage engine. Therefore, before the backup ends, an applied checkpoint is obtained. After the instance is restored, during the application process, the GTID executed set constructed by the Transaction Slot is used for idempotent control to catch up with the cluster checkpoint.
The acquisition and application logic of the above checkpoints ensures that after the backup set is connected to the original cluster, it can integrate the protocol layer, replication layer, and data layer to finally achieve strict consistency between various parts of the system.

Originally published at https://www.alibabacloud.com.

Introducing Alibaba Cloud for Generative AI

Lara Lee — Fri, 18 Oct 2024 08:12:52 +0000

Introduction

In the quickly changing digital world, using advanced technologies like generative AI is essential for businesses wanting a competitive edge. Companies now must tackle the task of using complex AI models while dealing with large amounts of data and computing needs. Alibaba Cloud stands out in this area, providing a full-stack solution for generative AI applications. This solution includes many services that help organizations build and enhance foundation models but also adjust these models to meet specific business goals. The easy deployment of these AI tools on a solid infrastructure improves performance and efficiency, making it suitable for both new and established businesses in their transformation journeys. Therefore, companies that use Alibaba Cloud’s expertise can drive innovation, improve operations, and enhance customer experiences with the help of generative AI.

Overview of Generative AI

Fast growth in artificial intelligence has led to generative AI, which uses machine learning to make content like text, images, music, and videos. Key to this tech are foundation models (FMs), which are trained on large datasets to create sensible and fitting outputs. This move to use FMs is a major change from older AI systems, as it highlights being flexible and personalized based on what users want. Companies like Alibaba Cloud show how generative AI can transform things by offering a complete solution that covers everything from developing and improving models to using them as online services that can grow. This type of support not only improves performance and efficiency but also makes advanced technology more available, helping businesses of all sizes to create and improve customer experiences with smart solutions (Bernard Marr).

Importance of Cloud Computing in AI Development

Among many tech advancements, the connection between cloud computing and artificial intelligence (AI) has become key for innovation. By using cloud services, organizations can access big computing power needed for training complex foundation models (FMs) without the high costs of traditional in-house setups. This is especially important because AI applications require more data, and need scalable solutions that can adjust to changing workloads. Alibaba Cloud’s wide range of services offers a complete solution for generative AI, helping to build and refine these models while making it easier to deploy them as online services on specialized AI infrastructure. Also, the cloud setup boosts teamwork among different fields, enabling quick updates needed in a fast-changing tech environment, which speeds up business changes through new AI technology (Jian Wang).

Alibaba Cloud’s Infrastructure for Generative AI

Alibaba Cloud’s method for building its generative AI setup shows a clear awareness of the different needs in the AI field. This complete solution covers all parts of the generative AI process, starting from making foundation models (FMs) to adjusting and launching them as strong online services. This connected system helps improve models based on particular business needs and makes sure that performance and efficiency are enhanced with specialized AI resources. Because of this, businesses of any size or experience can use generative AI effectively. This capability helps create new customer experiences, leading to notable changes in business in a more competitive market. Therefore, Alibaba Cloud’s infrastructure serves as an important tool for companies looking to incorporate advanced generative AI technologies into their operations (Alibaba Cloud Intelligence GTS).

Overview of Alibaba Cloud’s AI Infrastructure

A complete understanding of Alibaba Cloud’s AI setup shows its smart design aimed at helping generative AI use in businesses of all sizes. This all-in-one solution includes creating and improving foundation models (FMs) and highlights the need for customization through fine-tuning to meet specific business needs. The smooth connection of these functions is backed by a strong base structure that boosts both performance and efficiency. Additionally, the infrastructure offers the flexibility for businesses to move from development to deployment, allowing for the creation of modern customer experiences that are smart and can grow. By using Alibaba Cloud’s AI infrastructure, companies can drive change and use generative AI technologies to remain competitive in a more digital market, ultimately improving their strategic position through technology use and innovation (Alibaba Cloud Intelligence GTS).

Purpose-built AI Hardware and Software

In the last few years, the growth of AI hardware and software made for specific purposes has changed how companies use generative AI technologies. Key to this change is Alibaba Cloud’s complete GenAI solution, which provides a range of services to help create and use foundation models (FMs) that meet unique business needs. By offering improved infrastructure, the platform boosts performance and efficiency and makes the whole AI process simpler-from the first model training to adjustments and real-time use as online services. This new method makes sure that businesses, no matter their size or development stage, can use generative AI to enhance customer experiences and promote important business changes. Therefore, combining specialized AI hardware with custom software comes out as an important factor in gaining a competitive edge in the complex world of digital transformation (Miao et al.).

Scalability and Flexibility of Services

In the fast-changing tech world, being able to meet specific business needs is very important for companies that want to use generative AI well. Alibaba Cloud’s complete solution shows this flexibility by offering wide-ranging services for developing foundation models, making it easy for businesses to create, improve, and launch these models. This all-in-one method is especially important in industries like oil and gas, where issues with limited data and algorithm independence can hurt efficiency. By allowing businesses to fine-tune models based on their specific needs, Alibaba Cloud boosts the scalability of AI solutions, as noted in (He LIU), and encourages a lively setting for new applications. Additionally, the adaptability of using these models on a specially designed AI platform supports ongoing changes within companies, helping them quickly adjust to shifts in the market and customer needs.

Performance Optimization Techniques

Making generative AI applications work well depends on using infrastructure that is efficient and can grow. Alibaba Cloud’s complete solution shows how to do this by giving a specially designed AI setup that boosts the whole process of building models to put them into use. This way helps make fine-tuning foundation models (FMs) for certain business needs easier and makes sure these models work well in real life. For example, using advanced methods for performance can significantly lower delays and raise output, which is especially important for applications that need real-time data handling. Also, using tactics like moving workloads between clouds, as mentioned with Sky Computing, helps manage resources that can change based on different demand patterns (Zhanghao Wu). As the need for smart customer experiences grows, methods that improve operational efficiency become critical for driving sustainable business change through generative AI (Xiaoyi Ren).

Security Measures in AI Deployment

In the area of generative AI use, having a complex way to ensure security is very important to reduce risks related to data integrity, privacy, and system weaknesses. Alibaba Cloud shows through its broad solutions that putting foundational models (FMs) into a safe setup is key for businesses wanting to use AI technologies well. The SecGenAI framework specifically highlights the need to protect data privacy and secure deployment in cloud-based GenAI applications, especially against dangers such as data leaks and attacks from adversaries (Christoforus Yoga Haryanto et al.). Additionally, putting security measures in mobile AIGC networks shows the need for immediate protection and keeping user data confidential. This allows for personalized AI services while tackling privacy issues (Minrui Xu et al., p. 1127–1170). Therefore, a strong security system not only strengthens AI uses but also builds trust, allowing organizations to create new customer experiences without risking safety or rules.

Foundation Models and Their Development

In the last few years, artificial intelligence has changed a lot due to the rise of foundational models (FMs), which are important for generative AI applications. These models can be used for many different tasks, giving businesses the ability to customize AI solutions for their requirements. Alibaba Cloud’s full-stack solution plays a crucial role here, as it makes the whole FM development process easier, from building to deploying. By using specially designed AI infrastructure, organizations can improve model performance and efficiency, allowing them to create smarter customer experiences. This approach also makes advanced AI capabilities more available, so companies of all sizes can start important changes in their fields. As a result, the comprehensive support from Alibaba Cloud makes it a major player in the growth and use of foundational models in generative AI technology (Harold Booth)(Murugiah Souppaya).

Definition and Importance of Foundation Models

Improvements in artificial intelligence have led to the arrival of foundation models (FMs), which act as flexible structures for many tasks, changing how we see AI uses. These large models, trained on many different types of data, can fit into various areas with few changes needed for specific tasks. The value of FMs comes from their effectiveness and ability to grow, as well as their role in making advanced AI technology available to different organizations. Using one main structure, companies can gain insights and abilities that would usually need a lot of resources and knowledge to develop on their own. Companies like Alibaba Cloud see this significant change, offering customized solutions that help with the improvement, adjustment, and use of FMs to suit particular business requirements (Anand Vemula). Therefore, FMs signify not just a tech development but also a major change in how generative AI can promote new business methods.

Alibaba Cloud’s Tools for Building Foundation Models

The effect of new technology on business change is more clear, especially with the rise of generative AI apps. In this setting, Alibaba Cloud provides a full solution meant for building foundation models (FMs). This service covers everything in the AI model creation process, starting with building and improving to fine-tuning for specific business needs and easy deployment as online services. This organized method is important because it helps companies use strong AI tools for good performance and efficiency. By making these processes easier, Alibaba Cloud supports businesses of all sizes to change customer experiences and create big changes with generative AI technologies. The focus on flexibility and integration highlights the platform’s role in enabling new developments in various industries, making it an important player in the changing AI field (Anand Vemula).

Fine-tuning Foundation Models for Specific Applications

The development of foundation models (FMs) has changed the world of artificial intelligence applications, especially in specific fields. Alibaba Cloud’s all-inclusive generative AI (GenAI) solution shows how fine-tuning these models can help meet certain operational needs. By using AI systems designed for specific purposes, companies can easily adjust FMs, making them generate outputs that closely fit their individual goals. For example, recent innovations like TrafficLLM show how self-refinement methods can improve predictions of wireless traffic in changing situations, highlighting how important FMs are for solving complex prediction problems (Chengming Hu et al.). Additionally, the creation of Generative EEG Transformers shows how transformer-based structures can be fine-tuned to produce continuous neural signals for brain-computer interfaces, allowing for new uses in real-time data analysis (Omair Ali et al.). Through these instances, Alibaba Cloud’s GenAI highlights how fine-tuned FMs can transform sectors by increasing both accuracy and efficiency.

Deployment and Integration of Generative AI Solutions

The fast growth of generative AI technology has made it important to have strong systems for its use and integration in business environments. Alibaba Cloud offers a full-stack solution that simplifies this process by providing services for the whole lifecycle of foundation models (FMs). This all-encompassing approach helps in building and improving FMs and also enables fine-tuning to meet specific business needs, which is crucial for ensuring relevance and efficiency. These features are key in an environment where AI’s role is more important for customer interaction and business changes, especially given the claim that artificial intelligence is the key technology of our time (Shabbir Merali et al.). Also, as companies deal with the challenges of using AI solutions, Alibaba Cloud’s scalable structure helps address issues related to managing workloads and optimizing resources, both of which are vital for enhancing performance and cost efficiency (Zhanghao Wu).

Steps for Deploying Generative AI Models

Starting the use of generative AI models needs a clear plan to make sure they work well and fit with business goals. The first part is to carefully set the project scope by figuring out specific use cases and creating clear performance measures. Next, teams should choose the right foundation models (FMs) that can be adjusted to meet these specific needs, using Alibaba Cloud’s complete solution for optimizing these models. After that, it’s important to fine-tune the chosen models to make sure the AI’s results match business needs and improve its understanding of the context. Finally, in the deployment phase, teams should use Alibaba Cloud’s specially designed AI infrastructure to ensure the best performance and efficiency. By following these organized steps, businesses can handle the challenges of adding generative AI, leading to innovation and better customer experiences in lasting ways (Bernard Marr).

Integration with Existing Business Systems

The integration of Alibaba Cloud’s generative AI products with current business systems is important for digital transformation. Companies often deal with different systems that make it hard to use new technologies effectively, so it is vital to check compatibility and work together. By using Alibaba Cloud’s complete solution for generative AI, companies can not only improve base models but also adjust and implement them to fit their specific operations. This integration helps businesses to take advantage of advanced AI features across different functions, improving decision-making, enhancing customer interactions, and encouraging new ideas. Additionally, the infrastructure provided is built for good performance and efficiency, allowing businesses to grow their AI projects without high costs. In the end, creating smart customer experiences and driving major business plans relies on the successful integration with older systems, giving companies a better chance to compete in a fast-changing digital world (Alibaba Cloud Intelligence GTS).

Monitoring and Maintenance of Deployed Models

To make sure deployed models work well, it is important to keep an eye on them and maintain them, key parts that can influence how generative AI applications succeed. This process needs regular checking of model results to spot any differences from what was expected, as changes in the environment and user behavior can affect their performance. To lessen the risks that come with data changes and model deterioration, companies need to set up a strong framework with real-time monitoring tools and automated retraining methods. The rise of generative AI requires extra attention since its uses are broad, significantly changing how customers experience services and how businesses operate. As stated, “Artificial Intelligence will be the transformational technology of our generation” (Shabbir Merali et al.), highlighting the need for a strong infrastructure to support ongoing improvements to models. Additionally, dealing with the challenges of mixed sensory data using advanced methods is crucial for ensuring reliability in infrastructure applications (Pan He).

Business Transformation through Generative AI

The rise of generative AI is changing how businesses use technology to improve efficiency and customer experience. By using platforms like Alibaba Cloud, companies can find a complete solution for generative AI that meets different needs, from creating foundation models to optimizing and deploying on specific infrastructure. This all-in-one approach allows businesses to use AI effectively, which leads to new ways to engage customers and make processes smoother. For example, generative AI can create personalized content that boosts customer interaction, supporting research that shows that customer participation is influenced by the trustworthiness and skill of service providers (Yaozhi Zhang). Also, incorporating advanced AI technologies like deep learning meets the critical need for processing power, helping smart applications grow in many industries, which promotes change (Danlin Xu). Altogether, these developments show how vital generative AI is in changing business models and preparing companies for future success.

Impact of Generative AI on Business Processes

As businesses are using more digital change, generative AI technologies are bringing big improvements to how they operate and engage with customers. Alibaba Cloud’s complete solution for generative AI helps companies use foundation models (FMs) made for their needs, allowing them to react quickly to market changes and customer wants. This feature not only helps improve FMs but also supports easy use across different business tasks, changing how services are given. Additionally, advancements in intelligent photonics show that adding AI to business operations can lower energy use and increase processing speeds, which are important in today’s competitive environment (Danlin Xu). Nevertheless, it’s important to think about the impact of Big Tech in leading AI research and its uses, since their power influences how innovation and ideas spread in the industry (Stanisław Giziński). By using Alibaba Cloud’s generative AI solutions, businesses can move through this changing environment, finding new ways for growth and innovations focused on customers.

Innovations Driven by Generative AI

The way technology has changed is seen in the rise of generative AI, which helps drive new ideas across many areas. By using advanced tools for creating and improving content, companies are finding new chances to boost customer interaction and operational productivity. Alibaba Cloud’s wide-ranging system supports this shift by providing a complete solution for generative AI that not only helps in creating basic models but also aligns them with particular business goals. Current conversations point out that these new technologies are key to changing how we manage city traffic, where the use of Artificial Intelligence of Generative Content (AIGC) improves decision-making and optimizes traffic solutions (Xiaoyi Ren). Additionally, the economic effects and safety issues related to generative AI advancements are crucial, highlighting the need for careful regulations to protect both users and the quality of new technologies (Shabbir Merali et al.). This all-encompassing strategy helps explore the full potential of generative AI, leading to sustainable development and creating new expansion opportunities.

Read All
Originally published at https://www.alibabacloud.com.

How to Safeguard Your Cloud: Best Practices for Security and Compliance

Lara Lee — Thu, 17 Oct 2024 05:55:57 +0000

As small and medium-sized businesses (SMBs)increasingly adopt cloud computing, security and compliance become critical. The growing number of cyber threats, combined with evolving regulatory standards, make it essential for businesses to protect their data while ensuring they meet compliance requirements. This blog outlines key practices for cloud security and compliance, with real-world examples, offering valuable insights for SMBs looking to enhance their cloud security.

The Growing Cloud Threat Landscape

Cloud environments face various security threats, including data breaches, DDoS attacks, and insider threats. For SMBs, these risks can be particularly damaging, as they often lack the resources of larger organizations to recover from breaches or disruptions. Preventing security issues starts with addressing misconfigurations and strengthening access controls to limit unauthorized access.
Alibaba Cloud offers several solutions that can be leveraged to protect their cloud environments, including:
● Anti-DoS: Automatically mitigates large-scale DDoS attacks, protecting against service disruptions.
● Web Application Firewall (WAF): Prevents common web-based attacks and automatically patches vulnerabilities through virtual patching.
● Security Center: Provides continuous monitoring for vulnerabilities, configuration weaknesses, and intrusions, with a unified security management system.

Cloud Compliance: A Non-Negotiable Priority

For small businesses, achieving and maintaining compliance with regulatory standards such as GDPR, HIPAA, or PCI DSS is crucial to avoid penalties and build customer trust. However, navigating compliance can be overwhelming, particularly for SMBs operating across multiple cloud platforms or in hybrid environments. Automating compliance checks and audits can reduce this burden, ensuring businesses stay on top of regulatory requirements without expending significant time and resources.
Alibaba Cloud offers several tools to simplify compliance and security for SMBs:
● eKYC: An electronic Know Your Customer solution that streamlines identity verification, ensuring regulatory compliance in finance-related industries.
● CAPTCHA: Enhances security by preventing automated attacks like credential stuffing and bot-driven intrusions.

Best Practices for Cloud Security

To help SMBs safeguard their cloud environments, here are some key security measures:
● Data Encryption: Encrypting data both at rest and in transit protects sensitive information, a must for any business handling customer data.
● Identity and Access Management (IAM): Implement multi-factor authentication (MFA) and enforce least-privilege access controls to minimize the risk of insider threats.
● Threat Detection and Monitoring: Utilize automated tools to continuously monitor for vulnerabilities and potential attacks, ensuring real-time responses.
Alibaba Cloud’s Content Moderation solution also helps SMBs that handle user-generated content to automatically detect and remove inappropriate or harmful content, maintaining platform integrity and security.

Case Study 1: GCash’s Approach to Security and Compliance

GCash, a FinTech leader in the Philippines, faced challenges scaling its infrastructure while ensuring security. Like many SMBs, GCash initially relied on legacy on-premise systems, which caused downtimes and security concerns as the company grew. By partnering with Alibaba Cloud, GCash implemented solutions such as Anti-DDoS, Web Application Firewall (WAF), Security Center, and eKYC. These tools allowed GCash to manage a growing user base and defend against cyber threats while staying compliant.
GCash followed three core principles, which serve as valuable lessons for SMBs:

Develop in Grayscale: Gradually roll out changes to minimize disruption.
Monitor Deployments: Regularly track system performance to catch issues early.
Create a Contingency Plan: Always have a recovery plan in place to mitigate unforeseen issues. This approach helped GCash handle a 1,000% surge in transactions during the pandemic, with Alibaba Cloud’s built-in security tools safeguarding its infrastructure from threats.

Case Study 2: AirAsia's Battle Against Bots and Security Threats

AirAsia, another SMB example in the airline industry, heavily relies on its web-based ticketing platform. However, the platform was vulnerable to attacks from bots and scrapers, which not only caused performance issues but also increased operational costs. AirAsia collaborated with Alibaba Cloud to implement CDN and Web Application Firewall (WAF) solutions to optimize performance and enhance security.
Through their partnership, AirAsia was able to identify and block 90% of bot traffic, improving system performance and reducing risks to customer data. Alibaba Cloud's team provided weekly security reports and updates to ensure AirAsia stayed ahead of emerging threats. The deployment of CAPTCHA further helped distinguish real users from bots, protecting the company’s revenue and customer experience.

Leveraging Cloud Providers for Security and Compliance

For SMBs, partnering with a reliable cloud provider like AWS, Azure, or Alibaba Cloud can significantly ease the burden of maintaining security and compliance. Providers offer ready-made solutions such as ZOLOZ for secure user authentication, which can help SMBs reduce operating costs while enhancing security. Understanding the shared responsibility model between cloud providers and customers is key to ensuring SMBs have the right protection in place without overburdening their IT teams.
As seen in GCash and AirAsia’s cases, partnering with the right cloud provider and adopting best practices for security and compliance can help SMBs scale effectively while protecting their most critical assets—data, and user trust. Whether you're just starting your cloud journey or looking to optimize your existing setup, prioritizing security and compliance will position your business for long-term success.

Alibaba Cloud’s AI Technology Sparks Breakthrough in RNA Virus Discovery

Lara Lee — Tue, 15 Oct 2024 03:40:40 +0000

The tool led to the discovery of +160,000 potential RNA virus species and 180 RNA virus supergroups.
Deep learning algorithms can be applied to fields like microbial genomics and epidemiology

How we identify new viruses is about to get faster and more precise, thanks to groundbreaking research led by Alibaba Cloud in collaboration with China’s Sun Yat-sen University and The University of Sydney in Australia.

In a study published 01085–7) on Oct. 9 in the peer-reviewed scientific journal Cell, the trio unveiled a deep-learning algorithm that uses artificial intelligence to detect RNA viruses.

LucaProt, as the model is known, improves the viral discovery process by analyzing samples for protein sequences and structural features, helping researchers discover thousands of potential viruses.

“The success of this model not only paves the way for further applications of artificial intelligence in microbiology but also sets a new standard for data-driven discovery in biological sciences,” said Yong He, algorithm expert at Alibaba Cloud’s Apsara Lab and co-author of the recent research paper.

The study is the largest virus species discovery ever published in terms of the number of species identified and has given rise to a powerful healthcare tool.

LucaProt has excellent accuracy (0.014% false positives) and specificity (1.72% false negatives) on test data sets, leading to the discovery of 160,000 potential RNA virus species and 180 RNA virus supergroups.

“This research is part of Alibaba Cloud’s ongoing efforts to collaborate with academic and research institutions to harness the power of cloud computing and AI in driving the innovation and discovery in science research,” Yong added.

A Major Milestone

RNA viruses are the cause of certain diseases which makes understanding the evolution of these viruses vital for public health.

But traditional methods of virus identification that rely on sequence similarity can overlook unknown viruses, putting an entire realm of viruses out of reach — until now.

During the study, researchers applied LucaProt to a data set of 10,487 metatranscriptomes from diverse ecological systems and watched as the algorithm identified 161,979 potential new virus species and 180 new RNA virus supergroups.

The newly discovered viruses were present in a range of environments, from air to hot springs to hydrothermal vents, and virus diversity and abundance varied substantially among them all.

“We have been offered a window into an otherwise hidden part of life on earth, revealing remarkable biodiversity,” said Edward Holmes, a senior author on the paper and a professor at the School of Medical Sciences in the Faculty of Medicine and Health at the University of Sydney.

Applications and Implications

Leveraging AI, researchers are able to go beyond the constraints of traditional methods to embrace a more dynamic, comprehensive, and precise approach to virus discovery.

“We used to rely on tedious bioinformatics pipelines for virus discovery, which limited the diversity we could explore. Now, we have a much more effective AI-based model that offers exceptional sensitivity and specificity, and at the same time allows us to delve much deeper into viral diversity,” paper co-author Mang Shi said.

With new viruses being discovered in some of the most extreme environments on the planet, it is clear that RNA viruses are not only more diverse than previously thought but also more resilient and adaptable.

“To find this many new viruses in one fell swoop is mind-blowing, and it just scratches the surface, opening up a world of discovery. There are millions more to be discovered, and we can apply this same approach to identifying bacteria and parasites,” Holmes said.

To aid this mission, Alibaba Cloud has unveiled several AI-driven discovery tools, which can identify patterns and features in complex data in fields like microbial genomics and epidemiology.

The specialized models include a unified base model for nucleic acids and proteins named LucaOne, and a protein language model named LucaPcycle.

This article was originally published on Alizila, written by Elizabeth Utley and Alizila Staff. You may also read this article at https://www.alibabacloud.com/?utm_content=g_1000396071.

Best Practices for GanosBase Real-Time Heatmap Aggregation Query

Lara Lee — Sat, 12 Oct 2024 09:51:49 +0000

This article introduces GanosBase's real-time heatmap aggregation query and dynamic heatmap tile output capabilities, built on Alibaba Cloud's PolarDB for PostgreSQL.

About GanosBase

Best Practices for GanosBase Real-Time Heatmap Aggregation Query
About GanosBase
GanosBase is a next-generation cloud-native spatial intelligence engine jointly developed by Alibaba Cloud Database Product Division and the Feitian Database and Storage Laboratory. It integrates spatiotemporal data processing capabilities into core products such as the cloud-native relational database PolarDB for PostgreSQL, cloud-native multi-model database Lindorm, cloud-native data warehouse AnalyticDB for PostgreSQL, and cloud database ApsaraDB RDS for PostgreSQL. GanosBase currently features ten core engines: Geometry, Raster, Trajectory, Surface Mesh, Volume Mesh, 3D Scene, Point Cloud, Networking, Geometry Grid, and Fast Display, providing databases with integrated capabilities for storage, query, analysis, and service of new physical world multi-modal and polymorphic data.
This document introduces GanosBase's real-time heatmap aggregation query and dynamic heatmap tile output capabilities, built on Alibaba Cloud's PolarDB for PostgreSQL.

About Heatmap Tiles

What are Heatmap Tiles

Heatmap Tiles (HMT) are based on GanosBase's pioneering real-time heatmap aggregation query technology for large-scale vector/trajectory data. They are used to immediately return query results to the client, changing the traditional method of heatmap statistical analysis that required pre-coding for aggregation and pre-slicing for display. HMT can aggregate and render data on a scale of millions, tens of millions, and even hundreds of millions in seconds. HMT supports various commonly used aggregation functions and algebraic expressions, allowing customers to choose indicators of interest for their business and dynamically compute and render at different levels as the map zooms in and out. This greatly enhances business efficiency and offers more possibilities for customer products. At this year's Yunqi Conference, GanosBase released this capability and demonstrated a case of real-time query aggregation of large-scale transportation trajectories using HMT, helping customers to fully online data products that previously required offline preprocessing. This feature has received significant industry recognition and approval.

Use Cases for Heatmap Tiles

The key feature of Heatmap Tiles is real-time aggregation and rendering of spatial data, primarily applied in business scenarios with massive amounts of vector data requiring real-time statistical analysis, such as:

1. Transportation: Aggregate real-time heatmaps across entire regions based on the historical trajectory lines of transportation tools (cars, ships, etc.) and filter them by conditions such as time (winter, summer), start/end points, and type (freight, passenger) to generate corresponding heatmaps in real-time.

2. Urban Management: Based on building footprint data, aggregate single indicators such as building density, average building height, and total building area across entire regions and combine them with land parcel information to calculate composite indicators like floor area ratio.

3. Shared Mobility: Aggregate heatmaps of equipment docking areas across entire regions based on the trajectory points of shared mobility devices, and analyze shared mobility equipment's dispatch and operations strategies based on events (unlocking/locking, boarding/alighting, accidents, damage), flow directions, etc.

Technical Advantages of Heatmap Tiles

Compared to the pre-coding and aggregation method based on H3 or S2 grids, HMT Heatmap Tiles offer the following advantages:

1. Extremely efficient, no pre-coding required, and no additional storage costs.
HMT's aggregation technology differs from grid aggregation methods like H3/S2 in technical characteristics and application scenarios. Grid aggregation methods are often used when grid coding is needed as a retrieval condition because they require a predetermined precision level. The vector data is then coded at that precision level and aggregated based on the coding. However, HMT does not require pre-coding of data and can aggregate based on the current viewport range. As the viewport zooms in and out, aggregation is performed in real time. The entire process is equally efficient for various types of geometries, capable of aggregating and rendering data on a scale of hundreds of millions in seconds.

2. Convenient and easy to use, aggregation results can be directly visualized.
HMT offers the capability to quickly tile the aggregation results, allowing direct visualization integration with front-end rendering engines to ensure what you see is what you get. HMT also provides a series of statistical functions to help users quickly and automatically generate the best rendering color tables, ensuring optimal front-end performance.

Tested in real-world scenarios by multiple customers, HMT's aggregation efficiency is extremely high, typically achieving full map aggregation of hundreds of millions of data points in seconds:

Note: The above data represents full aggregation efficiency at a global display scale; efficiency increases as the map zooms in.

Functionality of Heatmap Tiles

Heatmap Tiles include a series of SQL functions for generating and calculating heatmap tiles, including:
● ST_AsHMT: Converts a set of geometric or trajectory objects into heatmap matrix tiles according to the specified range and resolution.
● ST_HMTAsArray: Converts heatmap tiles into an array matrix representation for easy viewing.
● ST_HMTStats: Calculates statistical information for heatmap tiles.
● ST_HMTAsRaster: Converts heatmap tiles into Raster objects for viewing and computation.

Best Practices for Heatmap Tiles

Operational Steps

Import geometric or trajectory data into the database, preferably using the FDW method. Ensure that all objects share the same spatial reference system (this can be confirmed with the ST_Srid function).
Create spatial indexes for the geometry or trajectory columns:
CREATE INDEX index_name ON table_name USING GIST(column_name)
Query heatmap tiles based on the spatial range:
Aggregate the number of objects within the grid:

SELECT ST_AsHMT(column_name, --geometry type ST_MakeEnvelope(0, 0, 10, 10, 4326), -- Extent 512, -- Width 512 -- height ) FROM table_name WHERE column_name && ST_MakeEnvelope(0, 0, 10, 10, 4326);

You can also aggregate the values within a grid by summing the values in the value column:

SELECT ST_AsHMT(column_name, --geometry type ST_MakeEnvelope(0, 0, 10, 10, 4326), -- Extent 512, -- Width 512, -- height value -- value column ) FROM table_name WHERE column_name && ST_MakeEnvelope(0, 0, 10, 10, 4326);

The ST_MakeEnvelope function can use the ST_TileEnvelope function to obtain the tile range. Additional filtering conditions can also be added:
SELECT ST_AsHMT(column_name, --geometry type ST_MakeEnvelope(0, 0, 10, 10, 4326), -- Extent 512, -- Width 512, -- height value -- value column ) FROM table_name WHERE column_name && ST_MakeEnvelope(0, 0, 10, 10, 4326); AND name like 'xxxx%' AND value > 100;

Usage Tips

● Improve performance with parallel processing when dealing with large datasets, as shown in the example below with a parallelism level of 16:
SET max_worker_processes = 300; -- Maximum number of background processes set max_parallel_workers = 260; -- maximum number of workers set max_parallel_workers_per_gather = 16; --maximum number of workers that can be started by a single Gather or Gather Merge node alter table table_name set (parallel_workers=16); set force_parallel_mode = on;

In practice, you can adjust settings based on the viewport range, such as using 16 parallel processes at higher levels and no parallelism at lower levels.

If sufficient CPU resources are available, to ensure that each query can utilize parallel processing, set max_worker_processes and max_parallel_workers to the product of parallelism and concurrency. Refer to the official PostgreSQL documentation.

● Tile Size
In most cases, use 512x512 tiles and resample to 256x256 to avoid aliasing issues. In special cases, such as when data volume is extremely large and each tile calculation is time-consuming, using larger tiles (1024x1024) can reduce the number of tile fetches and improve performance.

● Use the Operator for Spatial Filtering
Since ST_AsHMT computation is much faster than ST_Intersects, use && instead of ST_Intersects for index filtering. SELECT ST_AsHMT(column_name, --geometry type ST_MakeEnvelope(0, 0, 10, 10, 4326), -- Extent 512, -- Width 512, -- height value -- value column ) FROM table_name WHERE column_name && ST_MakeEnvelope(0, 0, 10, 10, 4326);

● Perform Spatial Reference Conversion on Query Range
When the query range and the geometry object's spatial range are inconsistent, perform spatial reference conversion on the query range before querying. Otherwise, automatic conversion may result in lower performance. After obtaining the tile, convert the image to the specified spatial reference for display.
SELECT ST_AsHMT(column_name, -- srid = 4326 ST_Transform(ST_TileEnvelope(6, 48, 32), 4326), -- Extent 512, -- Width 512, -- height value -- value column ) FROM table_name WHERE column_name && ST_Transform(ST_TileEnvelope(6, 48, 32), 4326));

● Perform VACUUM FULL and CLUSTER Operations on Spatial Tables
VACUUM operations reclaim free space and reduce disk file size, reducing IO during queries. CLUSTER operations align data organization with indexes, storing adjacent spatial data in adjacent data pages, and reducing disk access by the database.

VACUUM full table_name； Cluster table_name using index_name;

Publishing Services for Viewing

We will use Node.js to write a simple application demonstrating the actual use case of heatmap tiles.
File Structure

└── hmt_server ├── app.js ├── hmt. proto ├── index.html └── package.json

Where hmt. proto is the proto file introduced in the ST_AsHMT section above, and the contents of the other files will be provided below.

Backend Code

{ "name": "hmt_server", "version": "1.0.0", "main": "app.js", "license": "ISC", "dependencies": { "chroma-js": "^2.4.2", "express": "^4.18.2", "lru-cache": "^10.1.0", "pg": "^8.11.3", "protobufjs": "^7.2.5", "sharp": "^0.32.6" } }

const express = require('express');
const { Pool } = require('pg');
const chroma = require('chroma-js');
const sharp = require("sharp");
const protobuf = require('protobufjs');
const { LRUCache } = require('lru-cache');

// Set up database connection
const CONNECTION = {
  user: 'YOUR_USER',
  password: 'YOUR_PWD',
  host: 'YOUR_HOST',
  database: 'YOUR_DB',
  port: YOUR_PORT
};

// Target table name
const TABLE_NAME = 'YOUR_TABLE';

// Target geometry column name
const GEOMETRY_COLUMN = 'YOUR_GEOM_COLUMN';

// Set no data value
const NO_DATA_VALUE = 0;

// Target geometry column spatial reference
const SRID = 4326

// Set color map
const COLOR_MAP = [
  ['#536edb', 1],
  ['#5d96a5', 3],
  ['#68be70', 5],
  ['#91d54d', 7],
  ['#cddf37', 9],
  ['#fede28', 11],
  ['#fda938', 13],
  ['#fb7447', 15],
  ['#f75a40', 17],
  ['#f24734', 19],
  ['#e9352a', 21],
  ['#da2723', 23],
  ['#cb181d', 25]
];

// Create a database connection pool, default is 10 connections
const pool = new Pool(CONNECTION);

// Configure color conversion
const [colors, domains] = COLOR_MAP.reduce(([c, d], [colors, domains]) =>
  [[...c, colors], [...d, domains]], [[], []]);
const colorMap = chroma.scale(colors).domain(domains).mode('rgb')

// Load protobuf
const hmtDecoder = protobuf.loadSync('./hmt.proto').lookupType('HMT');

// Create a 1x1 transparent PNG to return as an empty tile
const emptyPng = Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAADUlEQVQImWP4//8/AwAI/AL+hc2rNAAAAABJRU5ErkJggg==', 'base64');

// For small-scale tiles (z<5), where updates are relatively minor, set a 24-hour cache expiration
const globalCache = new LRUCache({ max: 1000, ttl: 1000 * 3600 * 24 });

// For larger-scale tiles (z>=5), set a 12-hour cache expiration, adjustable based on actual conditions
const localCache = new LRUCache({ max: 2000, ttl: 1000 * 3600 * 12 });

// Register Express routes
express()
  // Serve the HTML page
  .get("/", (_, res) => res.sendFile('index.html', { root: __dirname }))
  // Serve heatmap tile service
  .get('/hmt/:z/:x/:y', async ({ params: { z, x, y } }, res) => {
    const cache = z < 5 ? globalCache : localCache;
    const key = `${z},${x},${y}`
    if (!cache.has(key)) {
      // Set parallelism and call ST_AsHMT function to request a 256x256 heatmap tile for the area
      const parallel = z <= 5 ? 10 : 5;
      const sql = `
  set max_parallel_workers = ${parallel};
  set max_parallel_workers_per_gather = ${parallel};
  WITH _PARAMS(_BORDER) as (VALUES(ST_Transform(ST_TileEnvelope(${key}),${SRID})))
  SELECT ST_AsHMT(${GEOMETRY_COLUMN},_BORDER,256,256) tile
  FROM ${TABLE_NAME},_PARAMS
  WHERE _BORDER && ${GEOMETRY_COLUMN};`
      // Skip the set statement and get the result of the ST_AsHMT function
      const { rows: [{ tile }] } = (await pool.query(sql))[2];

      // If there is no data in the area, return an empty tile
      if (!tile) cache.set(key, emptyPng);
      else {
        // Parse protobuf result
        const { type, doubleValues, intValues } = hmtDecoder.decode(tile);
        const { values } = type == 1 ? doubleValues : intValues;

        // Convert values to corresponding colors and remove no data values
        const pixels = values.reduce((_pixels, value) => {
          _pixels.push(...colorMap(value).rgb());
          _pixels.push(value <= NO_DATA_VALUE ? 0 : 255);
          return _pixels;
        }, [])

        // Render as PNG tile
        const rawConfig = { raw: { width: 256, height: 256, channels: 4 } };
        const renderedPng = await sharp(Uint8Array.from(pixels), rawConfig)
          .png().toBuffer();
        cache.set(key, renderedPng);
      }
    }
    const tile = cache.get(key)
    res.set("Content-Type", "image/png").send(tile);
  })
  // Listen on port 5500
  .listen(5500, () => console.log('HMT server started.'));

Frontend Code
We use Mapbox as the front-end map SDK. You can apply for a token here. Since the heatmap tiles are finally rendered in PNG format, they are compatible with most other map SDKs.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>HMT Viewer</title>

  <meta name="viewport" content="initial-scale=1,maximum-scale=1,user-scalable=no">
  <link href="https://api.mapbox.com/mapbox-gl-js/v2.14.1/mapbox-gl.css" rel="stylesheet">
  <script src="https://api.mapbox.com/mapbox-gl-js/v2.14.1/mapbox-gl.js"></script>

</head>

<body>
  <div id="map" style="position: absolute;left:0; top: 0; bottom: 

0; width: 100%;"></div>

  <script>
    let CENTER = [YOUR_LONGITUDE, YOUR_LATITUDE]
    mapboxgl.accessToken = YOUR_MAPBOX_TOKEN;
    const map = new mapboxgl.Map({
      container: 'map',
      style: "mapbox://styles/mapbox/navigation-night-v1",
      center: CENTER,
      zoom: 5
    })
    map.on("load", () => {
      map.addSource('hmt_source', {
        type: 'raster',
        minzoom: 3,
        tiles: [`${window.location.href}hmt/{z}/{x}/{y}`],
        tileSize: 256,
      });
      map.addLayer({
        id: 'hmt',
        type: 'raster',
        source: 'hmt_source',
      });
    });
  </script>

</body>

</html>

Installation and Deployment

# Navigate to the hmt_server directory
cd ./hmt_server
# Install dependencies
npm i
# Run the heatmap tile service
node .
# You can then open your browser and log in to http://localhost:5500/ to see the effect

Preview

Real-time Aggregation of Ship Trajectory Lines

31 million trajectory points, 450,000 trajectory lines real-time aggregation

Summary

Currently, GanosBase has evolved to version 6.0, supporting thousands of application scenarios across dozens of industries. Stability, cost-effectiveness, performance, and ease of use have always been GanosBase's long-term goals. HMT Heatmap Tiles represent GanosBase's core competitive edge in the efficient aggregation and visualization of large-scale spatial data. It provides customers with truly efficient and easy-to-use solutions for large-scale data analysis and mining, and we welcome all users to experience it.

Next Step

Start to build your heatmap application with PolarDB for PostgreSQL always a free program. Or you can follow our best practice to enable one in just a few clicks.

Originally published at https://www.alibabacloud.com.

Alibaba's AI-Powered Cancer Detection Tool Recognized on Fortune's 2024 Change the World List

Lara Lee — Fri, 11 Oct 2024 08:55:48 +0000

Fortune showcases 52 companies making significant social or environmental impacts worldwide
Alibaba's AI-powered tool outperforms radiologists in spotting pancreatic cancer in non-contrast CT scans

Fortune has named Alibaba Group to its 10th annual Change the World list, recognizing the world's top companies leading innovation and social impact through profit-driven operations.

A team of Fortune editors winnowed down a field of 250 nominees to craft the final list of 52 companies, which includes Alibaba in 8th place - the only Chinese company in the top 10.

"These companies let their actions speak for themselves-harnessing the creative impulses of capitalism to address social problems and generating revenue while doing so," Fortune Executive Features Editor Matt Heimer said in a statement.

Alibaba earned its spot for a groundbreaking tool developed by its research arm, DAMO Academy, that makes the diagnosis of cancerous malignancies faster and more cost-effective.

The AI-powered cancer detector screens for early signs of pancreatic ductal adenocarcinoma, which is responsible for nearly half a million deaths globally every year, according to estimates.

"Our goal is to leverage AI to cover the entire process of cancer treatment early detection to diagnosis with great precision and accessibility," said Le Lu, head of DAMO's medical AI team.

Researchers have dubbed the tool PANDA. It was deployed earlier this year at two hospitals in Zhejiang province, on China's east coast.

Researchers have dubbed the tool PANDA. It was deployed earlier this year at two hospitals in Zhejiang province, on China’s east coast.

PANDA is 34.1% more sensitive than radiologists to picking up abnormalities when it comes to screening scans, according to a study published in 2023 by DAMO Academy in collaboration with over 10 medical institutions.

The deep learning-based model can detect cancerous masses in the pancreas by examining non-contrast CT scans, a more efficient form of medical imaging used worldwide with a lower dose of radiation than contrast CT scans.

"Our cloud-based solution empowers patients who might not have access to cancer screening due to the scarcity or high costs associated with expert medical care," said Lu, who spoke with Alizila in May about the innovation.

Since launching in 2023, the tool's use has expanded from identifying pancreatic cancer to detecting a range of cancers, including tumors of the liver, esophagus, colon, and more.

"We'll continue to leverage technology and strategic partnerships with hospitals to make healthcare services more accessible, comprehensive, accurate, cost-effective, and efficient," added Lu.

Leveraging AI for Social Good

Alibaba's business units are deploying AI in a range of use cases to the benefit of wider society.

At last year's Asian Games, Alibaba Cloud launched a digital avatar named Xiaomo to facilitate two-way translation between sign language and Chinese.

The AI-powered translator took two years to develop and enabled hearing-impaired Asiad participants to ask people for directions, view event schedules, and more.

Earlier this year, the cloud computing company also introduced an AI-powered tool to create picture books for autistic children.

Since launching online in June, more than 50,000 people have used the feature, which generates audio-enhanced artbooks from a prompt fed into Alibaba's Qwen family of large language models.

"Technology can improve the quality of life with more convenient, equitable, and inclusive services," Sun Lijun, Alibaba Partner and Chairman of the Alibaba Foundation noted in Alibaba's Aspirations and Actions in Support of Disability 2023 report.

This article was originally published on Alizila, and written by Elizabeth Utley.

Originally published at https://www.alibabacloud.com.

Strengthen Your Cloud Security: Comprehensive Protection with Security Center

Lara Lee — Tue, 08 Oct 2024 07:17:04 +0000

Maintaining robust security for cloud resources and on-premises servers is crucial in today's digital landscape. Security Center is a centralized security management system that dynamically identifies, analyzes, and alerts you to security threats in real time. Leveraging big data technologies, Security Center offers a suite of features to protect your assets from ransomware, viruses, and web tampering while ensuring compliance with regulatory standards.

Key Benefits of Security Center

Unified Security Management

Threat Management for Your Entire Network:
The Security Center automatically collects various log data from your cloud services, implementing control over detected security threats. This unified approach ensures comprehensive threat management across your entire network.
Server Guard Management on Different Platforms:
The Security Center is compatible with multiple operating systems, including Windows, Linux, Ubuntu, and Debian. Whether deployed on Alibaba Cloud or third-party servers, the Security Center provides centralized control, allowing you to view information and perform operations in a unified web console regardless of the environment or region.
High Stability and High Availability:
The server-side control program, analysis and computing engines, and dependent middleware of the Security Center are deployed across multiple data centers. This deployment strategy ensures the high availability and stability of the system, providing reliable security management.

Proactive Defense Mechanisms

Anti-virus and Anti-ransomware:
The Security Center supports the proactive detection and termination of mainstream ransomware, mining programs, backdoor programs, worms, malicious programs, DDoS trojans, and trojan programs. This proactive defense mechanism ensures that your assets remain protected from a wide range of threats.
Anti-tamper:
The Security Center prevents websites from being maliciously implanted with content related to terrorism, politics, dark chains, trojans, and backdoors. This anti-tamper feature ensures the normal operation of web page information, protecting your online presence from malicious alterations.

Automated Security Operations

Automated Attack Traceability:
The Security Center automatically traces the sources and causes of attacks. This automated traceability helps you understand the origins and progression of intrusion threats, enabling quick and informed responses to security incidents.
Custom Alerts:
The Security Center supports the secure migration of third-party data to the cloud. It allows for real-time analysis, association, and aggregation of large amounts of logs, and supports custom alert rules. This flexibility ensures that your security operations are tailored to your specific needs.

Core Features of Security Center

Security Protection

Vulnerability Management and Fixes:
The Security Center identifies vulnerabilities in mainstream systems and software, allowing you to fix these vulnerabilities with just a few clicks. This streamlined process ensures that your systems remain secure and up to date.
Cloud Platform Configuration Assessment:
The Security Center links the capabilities of cloud services to form a security closed loop based on best security practices. This configuration assessment helps maintain a secure cloud environment.
Baseline Check:
The Security Center reduces risks caused by improper configurations by providing a checklist of the best configurations recommended by Alibaba Cloud. This baseline check ensures that your cloud resources are configured securely.

Threat Detection

Automated Alert Detection and Association:
The Security Center automatically associates alerts and identifies intrusions caused by low-risk anomalies. This automated detection improves operational efficiency by quickly highlighting potential threats.
Security Overview:
The Security Center provides a comprehensive security screen that displays network security metrics from multiple dimensions. This overview allows you to monitor and assess the security status of your network at a glance.

Investigation and Response

Log Analysis and Audit:
The Security Center offers robust log analysis and audit capabilities, providing a platform to trace attacks and ensure compliance. This feature is essential for maintaining a secure and compliant environment.
Asset Fingerprints:
The Security Center collects detailed information about your assets, including host port, process, software, and scheduled task information. This asset fingerprinting allows for thorough investigation and response to security incidents.

Conclusion

Security Center is an indispensable tool for organizations seeking to enhance their cloud security posture. With its unified security management, proactive defense mechanisms, automated security operations, and comprehensive features, the Security Center ensures the protection of cloud resources and servers in data centers. By leveraging big data technologies and the extensive experience of Alibaba Cloud in security defense, Security Center provides a reliable and efficient solution for managing and mitigating security threats.

Embrace Security Center to safeguard your digital assets and maintain a secure, compliant, and resilient cloud environment.

Originally published at https://www.alibabacloud.com/?utm_content=g_1000396071.