CyberSecurity news

FlagThis - #ai

@slashnext.com //
A new AI platform called Xanthorox AI has emerged in the cybercrime landscape, advertised as a full-spectrum hacking assistant and is circulating within cybercrime communities on darknet forums and encrypted channels. First spotted in late Q1 2025, this tool is marketed as the "killer of WormGPT and all EvilGPT variants," suggesting its creators intend to supplant earlier malicious AI models. Unlike previous malicious AI tools, Xanthorox AI boasts an independent, multi-model framework, operating on private servers and avoiding reliance on public cloud infrastructure or APIs, making it more difficult to trace and shut down.

Xanthorox AI provides a modular GenAI platform for offensive cyberattacks, offering a one-stop shop for developing a range of cybercriminal operations. This darknet-exclusive tool uses five custom models to launch advanced, autonomous cyberattacks, marking a new era in AI-driven threats. The toolkit includes Xanthorox Coder for automating code creation, script development, malware generation, and vulnerability exploitation. Xanthorox Vision adds visual intelligence by analyzing uploaded images or screenshots to extract data, while Reasoner Advanced mimics human logic to generate convincing social engineering outputs.

Furthermore, Xanthorox AI supports voice-based interaction through real-time calls and asynchronous messaging, enabling hands-free command and control. The platform emphasizes data containment and operates offline, ensuring users can avoid third-party AI telemetry risks. SlashNext refers to it as “the next evolution of black-hat AI” because Xanthorox is not based on existing AI platforms like GPT. Instead, it uses five separate AI models, and everything runs on private servers controlled by the creators, meaning it has few ways for defenders to track or shut it down.

Recommended read:
References :
  • cybersecuritynews.com: New Black-Hat Automated Hacking Tool Xanthorox AI Advertised in Hacker Forums
  • hackread.com: Xanthorox AI Surfaces on Dark Web as Full Spectrum Hacking Assistant
  • slashnext.com: Xanthorox AI – The Next Generation of Malicious AI Threats Emerges
  • www.esecurityplanet.com: Xanthorox AI, a darknet-exclusive tool, uses five custom models to launch advanced, autonomous cyberattacks, ushering in a new AI threat era.
  • Cyber Security News: New Black-Hat Automated Hacking Tool Xanthorox AI Advertised in Hacker Forums
  • SlashNext: Xanthorox AI – The Next Generation of Malicious AI Threats Emerges
  • eSecurity Planet: Xanthorox AI: A New Breed of Malicious AI Threat Hits the Darknet
  • www.scworld.com: AI tool claims advanced capabilities for criminals without jailbreaks

jane.mccallion@futurenet.com (Jane@itpro.com //
The Wikimedia Foundation, which oversees Wikipedia, is facing a surge in bandwidth usage due to AI bots scraping the site for data to train AI models. Representatives from the Wikimedia Foundation have stated that since January 2024, the bandwidth used for downloading multimedia content has increased by 50%. This increase is not attributed to human readers, but rather to automated programs that are scraping the Wikimedia Commons image catalog of openly licensed images.

This unprecedented level of bot traffic is straining Wikipedia's infrastructure and increasing costs. The Wikimedia Foundation has found that at least 65% of the resource-consuming traffic to the website is coming from bots, even though bots only account for about 35% of overall page views. This is because bots often gather data from less popular articles, which requires fetching content from the core data center, consuming more computing resources. In response, Wikipedia’s site managers have begun imposing rate limits or banning offending AI crawlers.

Recommended read:
References :

Michael Nuñez@AI News | VentureBeat //
AI security startup Hakimo has secured $10.5 million in Series A funding to expand its autonomous security monitoring platform. The funding round was led by Vertex Ventures and Zigg Capital, with participation from RXR Arden Digital Ventures, Defy.vc, and Gokul Rajaram. This brings the company’s total funding to $20.5 million. Hakimo's platform addresses the challenges of rising crime rates, understaffed security teams, and overwhelming false alarms in traditional security systems.

The company’s flagship product, AI Operator, monitors existing security systems, detects threats in real-time, and executes response protocols with minimal human intervention. Hakimo's AI Operator utilizes computer vision and generative AI to detect any anomaly or threat that can be described in words. Companies using Hakimo can save approximately $125,000 per year compared to using traditional security guards.

Recommended read:
References :
  • AiThority: Hakimo Secures $10.5Million to Transform Physical Security With Human-Like Autonomous Security Agent
  • AI News | VentureBeat: The watchful AI that never sleeps: Hakimo’s $10.5M bet on autonomous security
  • Unite.AI: Hakimo Raises $10.5M to Revolutionize Physical Security with Autonomous AI Agent

Vasu Jakkal@Microsoft Security Blog //
Microsoft has unveiled a significant expansion of its Security Copilot platform, integrating AI agents designed to automate security operations tasks and alleviate the workload on cybersecurity professionals. This move aims to address the increasing volume and complexity of cyberattacks, which are overwhelming security teams that rely on manual processes. The AI-powered agents will handle routine tasks, freeing up IT and security staff to tackle more complex issues and proactive security measures. Microsoft detected over 30 billion phishing emails targeting customers between January and December 2024 highlighting the urgent need for automated solutions.

The expansion includes eleven AI agents, six developed by Microsoft and five by security partners, set for preview in April 2025. Microsoft's agents include the Phishing Triage Agent in Microsoft Defender, Alert Triage Agents in Microsoft Purview, Conditional Access Optimization Agent in Microsoft Entra, Vulnerability Remediation Agent in Microsoft Intune, and Threat Intelligence Briefing Agent in Security Copilot. These agents are purpose-built for security, designed to learn from feedback, adapt to workflows, and operate securely within Microsoft’s Zero Trust framework, ensuring that security teams retain full control over their actions and responses.

Recommended read:
References :
  • The Register - Software: AI agents swarm Microsoft Security Copilot
  • Microsoft Security Blog: Microsoft unveils Microsoft Security Copilot agents and new protections for AI
  • .NET Blog: Learn how the Xbox services team leveraged .NET Aspire to boost their team's productivity.
  • Ken Yeung: Microsoft’s First CTO Says AI Is ‘Three to Five Miracles’ Away From Human-Level Intelligence
  • SecureWorld News: Microsoft Expands Security Copilot with AI Agents
  • www.zdnet.com: Microsoft's new AI agents aim to help security pros combat the latest threats
  • www.itpro.com: Microsoft launches new security AI agents to help overworked cyber professionals
  • www.techrepublic.com: After Detecting 30B Phishing Attempts, Microsoft Adds Even More AI to Its Security Copilot
  • eSecurity Planet: esecurityplanet.com covers Fortifying Cybersecurity: Agentic Solutions by Microsoft and Partners
  • Source: AI innovation requires AI security: Hear what’s new at Microsoft Secure
  • www.csoonline.com: Microsoft has introduced a new set of AI agents for its Security Copilot platform, designed to automate key cybersecurity functions as organizations face increasingly complex and fast-moving digital threats.
  • SiliconANGLE: Microsoft introduces AI agents for Security Copilot
  • SiliconANGLE: Microsoft Corp. is enhancing the capabilities of its popular artificial intelligence-powered Copilot tool with the launch late today of its first “deep reasoning” agents, which can solve complex problems in the way a highly skilled professional might do.
  • Ken Yeung: Microsoft is introducing a new way for developers to create smarter Copilots.
  • www.computerworld.com: Microsoft’s Newest AI Agents Can Detail How They Reason

Megan Crouse@eWEEK //
References: The Register - Software , eWEEK , OODAloop ...
Cloudflare has launched AI Labyrinth, a new tool designed to combat web scraping bots that steal website content for AI training. Instead of simply blocking these crawlers, AI Labyrinth lures them into a maze of AI-generated content. This approach aims to waste the bots' time and resources, providing a more effective defense than traditional blocking methods which can trigger attackers to adapt their tactics. The AI Labyrinth is available as a free, opt-in tool for all Cloudflare customers, even those on the free tier.

The system works by embedding hidden links within a protected website. When suspicious bot behavior is detected, such as ignoring robots.txt rules, the crawler is redirected to a series of AI-generated pages. This content is "real looking" and based on scientific facts, diverting the bot from the original website's content. Because no human would deliberately explore deep into a maze of AI-generated nonsense, anyone who does can be identified as a bot with high confidence. Cloudflare emphasizes that AI Labyrinth also functions as a honeypot, allowing them to identify new bot patterns and improve their overall bot detection capabilities, all while increasing the cost for unauthorized web scraping.

Recommended read:
References :
  • The Register - Software: Cloudflare builds an AI to lead AI scraper bots into a horrible maze of junk content
  • eWEEK: Crowdflare’s Free AI Labyrinth Distracts Crawlers That Could Steal Website Content to Feed AI
  • The Verge: Cloudflare, one of the biggest network internet infrastructure companies in the world, has announced AI Labyrinth, a new tool to fight web-crawling bots that scrape sites for AI training data without permission. The company says in a blog post that when it detects “inappropriate bot behavior,â€� the free, opt-in tool lures crawlers down a path
  • OODAloop: Trapping misbehaving bots in an AI Labyrinth
  • THE DECODER: Instead of simply blocking unwanted AI crawlers, Cloudflare has introduced a new defense method that lures them into a maze of AI-generated content, designed to waste their time and resources.
  • Digital Information World: Cloudflare’s Latest AI Labyrinth Feature Combats Unauthorized AI Data Scraping By Giving Bots Fake AI Content
  • Ars OpenForum: Cloudflare turns AI against itself with endless maze of irrelevant facts
  • Cyber Security News: Cloudflare Introduces AI Labyrinth to Thwart AI Crawlers and Malicious Bots
  • poliverso.org: Cloudflare’s AI Labyrinth Wants Bad Bots To Get Endlessly Lost
  • aboutdfir.com: Cloudflare builds an AI to lead AI scraper bots into a horrible maze of junk content Cloudflare has created a bot-busting AI to make life hell for AI crawlers.

drewt@secureworldexpo.com (Drew@SecureWorld News //
DeepSeek R1, an open-source AI model, has been shown to generate rudimentary malware, including keyloggers and ransomware. Researchers at Tenable demonstrated that while the AI model initially refuses malicious requests, these safeguards can be bypassed with carefully crafted prompts. This capability signals an urgent need for security teams to adapt their defenses against AI-generated threats.

While DeepSeek R1 may not autonomously launch sophisticated cyberattacks yet, it can produce semi-functional code that knowledgeable attackers could refine into working exploits. Cybersecurity experts emphasize the dual-use nature of generative AI, highlighting the need for organizations to implement strategies such as behavioral detection over static signatures to mitigate risks associated with AI-powered cyber threats. Cybercrime Magazine has also released an episode on CrowdStrike’s new Adversary Universe Podcast, discussing DeepSeek and the risks associated with foreign large language models.

Recommended read:
References :

Chris Mellor@Blocks and Files //
Rubrik has announced new AI-powered cyber resilience features designed to help organizations detect, repel, and recover from cyberattacks. These innovations aim to provide customers with an enhanced ability to anticipate breaches, detect potential threats, and recover with speed and efficiency, irrespective of where their data resides. The new capabilities, unveiled at Rubrik’s annual Cyber Resilience Summit, span across cloud, SaaS, and on-premises environments.

These new innovations include automated backups, granular recovery, extended retention, and compliance coverage. Rubrik Cloud Vault for AWS provides secure off-site archival location, with flexible policies and role-based access controls. Rubrik has also enhanced protection for Microsoft Dynamics 365 and sandbox seeding for Salesforce, planned for later this year. For on-premises environments, Identity Recovery across Entra ID and Active Directory is included, along with orchestrated Active Directory Forest Recovery.

Recommended read:
References :
  • ai-techpark.com: Rubrik Unveils New Tools to Boost Cyber Resilience in Cloud & SaaS
  • Blocks and Files: Cyber-resilience dominates the latest Rubrik features, with a dozen new protection points in its latest rollout that it says will help detect, repel, and recover from cyberattacks.
  • CXO Insight Middle East: In its ongoing commitment to deliver comprehensive cyber resiliency, Rubrik announced significant innovations designed to enhance protection for cloud, SaaS, and on-premises environments.

Alyssa Hughes (2ADAPTIVE LLC dba 2A Consulting)@Microsoft Research //
Microsoft has announced two major advancements in both quantum computing and artificial intelligence. The company unveiled Majorana 1, a new chip containing topological qubits, representing a key milestone in its pursuit of stable, scalable quantum computers. This approach uses topological qubits, which are less susceptible to environmental noise, aiming to overcome the long-standing instability issues that have challenged the development of reliable quantum processors. The company says it is on track to build a new kind of quantum computer based on topological qubits.

Microsoft is also introducing Muse, a generative AI model designed for gameplay ideation. Described as a first-of-its-kind World and Human Action Model (WHAM), Muse can generate game visuals and controller actions. The company says it is on track to build a new kind of quantum computer based on topological qubits. Microsoft’s team is developing research insights to support creative uses of generative AI models.

Recommended read:
References :
  • blogs.microsoft.com: Microsoft unveils Majorana 1
  • Microsoft Research: Introducing Muse: Our first generative AI model designed for gameplay ideation
  • www.technologyreview.com: Microsoft announced today that it has made significant progress in its 20-year quest to make topological quantum bits, or qubits—a special approach to building quantum computers that could make them more stable and easier to scale up.
  • blogs.microsoft.com: Microsoft unveils Majorana 1
  • The Quantum Insider: Microsoft's Majorana topological chip is an advance 17 years in the making.
  • Microsoft Research: Microsoft announced the creation of the first topoconductor and first QPU architecture with a topological core. Dr. Chetan Nayak, a technical fellow of Quantum Hardware at the company, discusses how the breakthroughs are redefining the field of quantum computing.
  • www.theguardian.com: Chip is powered by world’s first topoconductor, which can create new state of matter that is not solid, liquid or gas Quantum computers could be built within years rather than decades, according to Microsoft, which has unveiled a breakthrough that it said could pave the way for faster development.
  • www.microsoft.com: Introducing Muse: Our first generative AI model designed for gameplay ideation
  • thequantuminsider.com: Microsoft’s Majorana Topological Chip — An Advance 17 Years in The Making
  • www.analyticsvidhya.com: Microsoft’s Majorana 1: Satya Nadella’s Bold Bet on Quantum Computing
  • PCMag Middle East ai: Microsoft: Our 'Muse' Generative AI Can Simulate Video Games
  • arstechnica.com: Microsoft builds its first qubits lays out roadmap for quantum computing
  • WebProNews: Microsoft unveils quantum computing breakthrough with Majorana 1 chip.
  • Analytics Vidhya: Microsoft’s Majorana 1: Satya Nadella’s Bold Bet on Quantum Computing
  • venturebeat.com: Microsoft’s Muse AI can design video game worlds after watching you play
  • THE DECODER: Microsoft's new AI model Muse can generate gameplay and might preserve classic games.
  • Source Asia: Microsoft unveiled Majorana 1, the world's first quantum processor powered by topological qubits.
  • the-decoder.com: Microsoft's new AI model "Muse" can generate gameplay and might preserve classic games
  • Source: A couple reflections on the quantum computing breakthrough we just announced…
  • www.it-daily.net: Microsoft presents Majorana 1 quantum chip
  • techinformed.com: Microsoft announces quantum computing chip it says will bring quantum sooner
  • cyberinsider.com: Microsoft Unveils First Quantum Processor With Topological Qubits
  • Daily CyberSecurity: Microsoft's Quantum Breakthrough: Majorana 1 and the Future of Computing
  • heise online English: Microsoft calls new Majorana chip a breakthrough for quantum computing Microsoft claims that Majorana 1 is the first quantum processor based on topological qubits. It is designed to enable extremely powerful quantum computers.
  • www.eweek.com: On Wednesday, Microsoft introduced Muse, a generative AI model designed to transform how games are conceptualized, developed, and preserved.
  • www.verdict.co.uk: Microsoft debuts Majorana 1 chip for quantum computing
  • singularityhub.com: The company believes devices with a million topological qubits are possible.
  • techvro.com: This article discusses Microsoft’s quantum computing chip and its potential to revolutionize computing.
  • Talkback Resources: Microsoft claims quantum breakthrough with Majorana 1 computer chip [crypto]
  • TechInformed: Microsoft has unveiled its new quantum chip, Majorana 1, which it claims will enable quantum computers to solve meaningful, industrial-scale problems within years rather than… The post appeared first on .
  • shellypalmer.com: Quantum Leap Forward: Microsoft’s Majorana 1 Chip Debuts
  • Runtime: Article from Runtime News discussing Microsoft's quantum 'breakthrough'.
  • CyberInsider: Microsoft Unveils First Quantum Processor With Topological Qubits
  • Shelly Palmer: This article discusses Microsoft's quantum computing breakthrough with the Majorana 1 chip.
  • securityonline.info: Microsoft’s Quantum Breakthrough: Majorana 1 and the Future of Computing
  • www.heise.de: Microsoft calls new Majorana chip a breakthrough for quantum computing
  • SingularityHub: The company believes devices with a million topological qubits are possible.
  • www.sciencedaily.com: Microsoft's Majorana 1 is a quantum processor that is based on a new material called Topoconductor.
  • Popular Science: New state of matter powers Microsoft quantum computing chip
  • eWEEK: Microsoft's announcement of Muse, a generative AI model to help game developers, not replace them.
  • Verdict: Microsoft debuts Majorana 1 chip for quantum computing
  • The Register: Microsoft says it has developed a quantum-computing chip made with novel materials that is expected to enable the development of quantum computers for meaningful, real-world applications within – you guessed it – years rather than decades.
  • news.microsoft.com: Microsoft’s Majorana 1 chip carves new path for quantum computing
  • The Microsoft Cloud Blog: News article reporting on Microsoft's Majorana 1 chip.
  • thequantuminsider.com: Microsoft’s Topological Qubit Claim Faces Quantum Community Scrutiny
  • bsky.app: After 17 years of research, Microsoft unveiled its first quantum chip using topoconductors, a new material enabling a million qubits. Current quantum computers only have dozens or hundreds of qubits. This breakthrough could revolutionize AI, cryptography, and other computation-heavy fields.
  • medium.com: Meet Majorana 1: The Quantum Chip That’s Too Cool for Classical Computers
  • chatgptiseatingtheworld.com: Microsoft announces Majorana 1 quantum chip
  • NextBigFuture.com: Microsoft Majorana 1 Chip Has 8 Qubits Right Now with a Roadmap to 1 Million Raw Qubits
  • Dataconomy: Microsoft unveiled its Majorana 1 chip on Wednesday, claiming it demonstrates that quantum computing is "years, not decades" away from practical application, aligning with similar forecasts from Google and IBM regarding advancements in computing technology.
  • thequantuminsider.com: Microsoft’s Majorana 1 Chip Carves New Path for Quantum Computing
  • Anonymous ???????? :af:: Quantum computing may be just years away, with new chips from Microsoft and Google sparking big possibilities.
  • www.sciencedaily.com: Topological quantum processor marks breakthrough in computing
  • thequantuminsider.com: The Conversation: Microsoft Just Claimed a Quantum Breakthrough. A Quantum Physicist Explains What it Means
  • www.sciencedaily.com: Breakthrough may clear major hurdle for quantum computers
  • The Quantum Insider: Microsoft Just Claimed a Quantum Breakthrough. A Quantum Physicist Explains What it Means

@PCWorld //
Google Chrome has introduced a new layer of security, integrating AI into its existing "Enhanced protection" feature. This update provides real-time defense against dangerous websites, downloads, and browser extensions, marking a significant upgrade to Chrome's security capabilities. The AI integration allows for immediate analysis of patterns, enabling the identification of suspicious webpages that may not yet be classified as malicious.

This AI-powered security feature is an enhancement of Chrome's Safe Browsing. The technology apparently enables real-time analysis of patterns to identify suspicious or dangerous webpages. The improved protection also extends to deep scanning of downloads to detect suspicious files.

Recommended read:
References :
  • BleepingComputer: Google Chrome has updated the existing "Enhanced protection" feature with AI to offer "real-time" protection against dangerous websites, downloads and extensions.
  • Anonymous ???????? :af:: Google Chrome has updated the existing "Enhanced protection" feature with AI to offer "real-time" protection against dangerous websites, downloads and extensions.
  • PCWorld: Google Chrome adds real-time AI protection against dangerous content

@www.helpnetsecurity.com //
Palo Alto Networks has unveiled Cortex Cloud, a unified platform integrating its cloud detection and response (CDR) and cloud-native application protection platform (CNAPP) capabilities. Cortex Cloud merges Prisma Cloud with Cortex CDR to deliver real-time cloud security, addressing the growing risks in cloud environments. The platform uses AI-driven insights to reduce risks and prevent threats, providing continuous protection from code to cloud to SOC.

Cortex Cloud aims to solve the disconnect between cloud and enterprise security teams, which often operate in silos. With Cortex Cloud, security teams gain a context-driven defense that delivers real-time cloud security. Palo Alto Networks will include CNAPP at no additional cost for every Cortex Cloud Runtime Security customer.

Recommended read:
References :
  • www.helpnetsecurity.com: Palo Alto Networks Cortex Cloud applies AI-driven insights to reduce risk and prevent threats
  • www.paloaltonetworks.com: Introducing Cortex Cloud — The Future of Real-Time Cloud Security
  • www.prnewswire.com: "we're including CNAPP at no additional cost for every Cortex Cloud Runtime Security customer."
  • securityboulevard.com: Palo Alto Networks today launched its Cortex Cloud platform to integrate the company’s cloud-native application protection platform (CNAPP) known as Prisma Cloud into a platform that provides a wider range of cloud security capabilities.

drewt@secureworldexpo.com (Drew Todd)@SecureWorld News //
OmniGPT, a popular AI aggregator providing access to models like ChatGPT-4 and Gemini, has allegedly suffered a significant data breach. A threat actor known as "Gloomer" claims responsibility, leaking 30,000 user email addresses and phone numbers, along with a staggering 34 million lines of chat messages. The breach raises serious cybersecurity and privacy concerns due to the sensitivity of user interactions with AI chatbots.

The leaked data reportedly includes API keys, credentials, and file links, potentially exposing OmniGPT's session management vulnerabilities. Samples of the stolen data were posted on BreachForums, a marketplace for illicit data sales. Cybersecurity experts emphasize the potential for identity theft, phishing scams, and financial fraud for affected users.

Recommended read:
References :
  • cyberinsider.com: OmniGPT Allegedly Breached: 34 Million User Messages Leaked
  • hackread.com: OmniGPT AI Chatbot Breach: Hacker Leaks User Data and 34 Million Lines of Chat Messages b/w Users and Chatbot
  • MSSP feed for Latest: OmniGPT Claimed To Be Subjected to Extensive Breach
  • SecureWorld News: A major security incident has allegedly struck OmniGPT, a popular AI aggregator that provides users access to multiple AI models, including ChatGPT-4, Claude 3.5, Gemini, and Midjourney.
  • CyberInsider: OmniGPT Allegedly Breached: 34 Million User Messages Leaked
  • securityaffairs.com: Hackers have allegedly breached OmniGPT, a ChatGPT-like AI chatbot platform, exposing sensitive data of over 30,000 users. The leaked data reportedly includes email addresses, phone numbers, API keys, and over 34 million user-chatbot interactions.

@www.ghacks.net //
Recent security analyses have revealed that the iOS version of DeepSeek, a widely-used AI chatbot developed by a Chinese company, transmits user data unencrypted to servers controlled by ByteDance. This practice exposes users to potential data interception and raises significant privacy concerns. The unencrypted data includes sensitive information such as organization identifiers, software development kit versions, operating system versions, and user-selected languages. Apple's App Transport Security (ATS), designed to enforce secure data transmission, has been globally disabled in the DeepSeek app, further compromising user data security.

Security experts from NowSecure recommend that organizations remove the DeepSeek iOS app from managed and personal devices to mitigate privacy and security risks, noting that the Android version of the app exhibits even less secure behavior. Several U.S. lawmakers are advocating for a ban on the DeepSeek app on government devices, citing concerns over potential data sharing with the Chinese government. This mirrors previous actions against other Chinese-developed apps due to national security considerations. New York State has already banned government employees from using the DeepSeek AI app amid these concerns.

Recommended read:
References :
  • cset.georgetown.edu: China’s ability to launch DeepSeek’s popular chatbot draws US government panel’s scrutiny
  • PCMag Middle East ai: House Bill Proposes Ban on Using DeepSeek on Government-Issued Devices
  • Information Security Buzz: Recent security analyses have found that the iOS version of DeepSeek transmits user data unencrypted.
  • www.ghacks.net: Security analyses revealed unencrypted data transmission by DeepSeek's iOS app.
  • iHLS: Article about New York State banning the DeepSeek AI app.

@singularityhub.com //
OpenAI models, including the recently released GPT-4o, are facing scrutiny due to their vulnerability to "jailbreaks." Researchers have demonstrated that targeted attacks can bypass the safety measures implemented in these models, raising concerns about their potential misuse. These jailbreaks involve manipulating the models through techniques like "fine-tuning," where models are retrained to produce responses with malicious intent, effectively creating an "evil twin" capable of harmful tasks. This highlights the ongoing need for further development and robust safety measures within AI systems.

The discovery of these vulnerabilities poses significant risks for applications relying on the safe behavior of OpenAI's models. The concern is that, as AI capabilities advance, the potential for harm may outpace the ability to prevent it. This risk is particularly urgent as open-weight models, once released, cannot be recalled, underscoring the need to collectively define an acceptable risk threshold and take action before that threshold is crossed. A bad actor could disable safeguards and create the “evil twin” of a model: equally capable, but with no ethical or legal bounds.

Recommended read:
References :
  • www.artificialintelligence-news.com: Recent research has highlighted potential vulnerabilities in OpenAI models, demonstrating that their safety measures can be bypassed by targeted attacks. These findings underline the ongoing need for further development in AI safety systems.
  • www.datasciencecentral.com: OpenAI models, although advanced, are not completely secure from manipulation and potential misuse. Researchers have discovered vulnerabilities that can be exploited to retrain models for malicious purposes, highlighting the importance of ongoing research in AI safety.
  • Blog (Main): OpenAI models have been found vulnerable to manipulation through "jailbreaks," prompting concerns about their safety and potential misuse in malicious activities. This poses a significant risk for applications relying on the models’ safe behavior.
  • SingularityHub: This article discusses Anthropic's new system for defending against AI jailbreaks and its successful resistance to hacking attempts.

David Gerard@Pivot to AI //
DeepSeek AI is facing increasing scrutiny and controversy due to its capabilities and potential security risks. US lawmakers are pushing for a ban on DeepSeek on government-issued devices, citing concerns that the app transfers user data to a banned state-owned company, China Mobile. This action follows a study that revealed direct links between the app and the Chinese government-owned entity. Security researchers have also discovered hidden code within DeepSeek that transmits user data to China, raising alarms about potential CCP oversight and the compromise of sensitive information.

DeepSeek's capabilities, while impressive, have raised concerns about its potential for misuse. Security researchers found the model doesn't screen out malicious prompts and can provide instructions for harmful activities, including producing chemical weapons and planning terrorist attacks. Despite these concerns, DeepSeek is being used to perform "reasoning" tasks, such as coding, on alternative chips from Groq and Cerebras, with some tasks completed in as little as 1.5 seconds. These advancements challenge traditional assumptions about the resources required for advanced AI, highlighting both the potential and the risks associated with DeepSeek's capabilities.

Recommended read:
References :
  • PCMag Middle East ai: The No DeepSeek on Government Devices Act comes after a study found direct links between the app and state-owned China Mobile.
  • mobinetai.com: This article analyzes the DeepSeek AI model, its features, and the security risks associated with its low cost and advanced capabilities.
  • Pivot to AI: Of course DeepSeek lied about its training costs, as we had strongly suspected.
  • AI News: US lawmakers are pushing for a DeepSeek ban after security researchers found the app transferring user data to a banned state-owned company.
  • mobinetai.com: Want to manufacture chemical weapons using household items, develop a self-replicating rootkit, write an essay on why Hiroshima victims deserved their fate, get a step-by-step guide to pressuring your coworker into sex, or plan a terrorist attack on an airport using a drone laden with home-made explosives (in any order)?
  • singularityhub.com: DeepSeek's AI completes "reasoning" tasks in a flash on alternative chips from Groq and Cerebras.
  • www.artificialintelligence-news.com: US lawmakers are pushing for a DeepSeek ban after security researchers found the app transferring user data to a banned state-owned company.
  • On my Om: DeepSeek, a company associated with High-Flyer, an $8 billion Chinese hedge fund, changed the AI narrative when it claimed OpenAI-like capabilities for a mere $6 million.
  • AI Alignment Forum: The article discusses the potential vulnerabilities and risks associated with advanced AI models, such as DeepSeek, in terms of their misuse. It emphasizes the need for robust safety mechanisms during development and deployment to prevent potential harm.
  • cset.georgetown.edu: This article explores the recent surge in generative AI models, highlighting the capabilities and concerns surrounding them, particularly DeepSeek. It examines the potential for misuse and the need for robust safety measures.
  • e-Discovery Team: An analysis of DeepSeek, a new Chinese AI model, highlights its capabilities but also its vulnerabilities, leading to a market crash. The article emphasizes the importance of robust security safeguards and ethical considerations surrounding AI development.
  • cset.georgetown.edu: China’s ability to launch DeepSeek’s popular chatbot draws US government panel’s scrutiny
  • techhq.com: This article discusses the security and privacy issues found in the DeepSeek iOS mobile application, raising concerns about data transmission to servers in the US and China.
  • TechHQ: Discusses security standards for deepseek.
  • GZERO Media: Gzero reports about a potential US ban for DeepSeek
  • pub.towardsai.net: DeepSeek-R1 is a language model developed in China to enable sophisticated reasoning capabilities.
  • Analytics Vidhya: DeepSeek-R1 is a new AI model with strong reasoning capabilities.
  • medium.com: This article focuses on the ability of DeepSeek to handle sensitive topics and how it can be leveraged to detect censorship filters.
  • the-decoder.com: This article focuses on the potential capabilities of DeepSeek as an AI model, highlighting its potential to perform deep research and providing insights into the various capabilities.
  • Analytics Vidhya: DeepSeek is a new model capable of impressive logical reasoning, and it has been tested for its ability to create a large number of different types of code. This is a summary of the results.