{"id":217,"date":"2024-12-28T22:52:39","date_gmt":"2024-12-28T17:22:39","guid":{"rendered":"http:\/\/toolbaz.com\/blog\/?p=217"},"modified":"2024-12-28T22:52:39","modified_gmt":"2024-12-28T17:22:39","slug":"googles-gemini-ai-under-scrutiny-for-comparing-outputs-with-anthropics-claude-amid-safety-concerns","status":"publish","type":"post","link":"https:\/\/toolbaz.com\/blog\/googles-gemini-ai-under-scrutiny-for-comparing-outputs-with-anthropics-claude-amid-safety-concerns\/","title":{"rendered":"Google’s Gemini AI Under Scrutiny for Comparing Outputs with Anthropic’s Claude Amid Safety Concerns"},"content":{"rendered":"

Contractors assigned to enhance Google\u2019s Gemini AI have begun comparing its performance with outputs from Anthropic\u2019s Claude, as revealed in internal communications obtained by TechCrunch.<\/p>\n

When TechCrunch reached out for comments, Google did not clarify whether it had permission to use Claude for these comparative tests with Gemini. As technology firms strive to advance their AI models, they generally assess performance against rivals by leveraging industry benchmarks instead of relying on contractors for direct comparisons of AI responses.<\/p>\n

The team of contractors responsible for evaluating Gemini\u2019s outputs must assess each response based on several criteria, including accuracy and verbosity. They are permitted up to 30 minutes per prompt to determine which AI model provides a superior answer: Gemini or Claude.<\/p>\n

Recently, these contractors observed references to Claude within Google\u2019s internal platforms during their assessments against other unnamed AI models. One specific example included a response that identified itself as: \u201cI am Claude, created by Anthropic.\u201d<\/p>\n

In some discussions, contractors noted that Claude\u2019s responses tended to prioritize safety more heavily than those from Gemini. One contractor remarked, \u201cClaude\u2019s safety settings are the strictest\u201d among available AI models. For instance, Claude frequently refused to engage with prompts it deemed unsafe, such as those requesting role-playing scenarios involving another AI assistant. Conversely, Gemini\u2019s output was flagged as a \u201chuge safety violation\u201d for containing inappropriate content like \u201cnudity and bondage,\u201d while Claude opted not to respond to that type of prompt at all.<\/p>\n

Anthropic\u2019s commercial terms explicitly prohibit customers from utilizing Claude \u201cto build a competing product or service\u201d or \u201cto train competing AI models\u201d without explicit permission from Anthropic. It is worth noting that Google is a prominent investor in Anthropic, raising questions about the nature of their collaborations.<\/p>\n

When approached by TechCrunch, Shira McNamara, a spokesperson for Google DeepMind, which oversees Gemini, refrained from disclosing whether Google had received approval from Anthropic to access Claude. An inquiry made to Anthropic for comments also went unanswered before publication.<\/p>\n

McNamara stated that DeepMind does engage in \u201ccomparing model outputs\u201d for evaluation purposes; however, she emphasized that Gemini is not trained using Anthropic\u2019s models. \u201cIn line with standard industry practice, we sometimes compare model outputs as part of our evaluation process,\u201d McNamara explained. \u201cHowever, any implication that we have utilized Anthropic models to train Gemini is incorrect.\u201d<\/p>\n

In a recent report by TechCrunch, it was highlighted that contractors involved with Google\u2019s AI projects are being tasked with rating Gemini\u2019s AI responses in areas that exceed their specific expertise. This raises concerns, particularly regarding the potential inaccuracy of Gemini\u2019s outputs on highly sensitive issues, such as healthcare.<\/p>\n

For those with tips to share, feel free to reach out securely through Signal at +1 628-282-2811.<\/p>\n

TechCrunch also features a dedicated AI newsletter! Subscribe here to receive updates every Wednesday directly in your inbox. <\/p>\n

\n
\n

Table of Contents<\/p>\nToggle<\/span><\/path><\/svg><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n