PHP Generator对象未从已处理的外部JSON API接收数据

arknldoa  于 12个月前  发布在  PHP
关注(0)|答案(1)|浏览(91)

我一直在使用这个resource作为参考，将基于Joomla 3的CLI脚本转换为基于Joomla 4/5的API脚本。该脚本从外部API检索新闻帖子，并将其作为单独的文章添加到Joomla。
我有$process函数工作。它成功地连接到外部API，并在使用print_r函数时在浏览器中输出数据数组。我在下面包含了完整的$process函数。

$process = function (string $givenHttpVerb, string $endPoint, string $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

字符串
使用$dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl, $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);调用$process函数
我有一个$generator函数，它接受$dataSourceResponse匿名函数和$apiUrl，这是基于Joomla的网站。$apiUrl是 * upgrade.domain.com/api/index.php/v1*。完整的$generator函数包含在下面。

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => $article->title,
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => $article->topic,
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
    }
    $dataString = json_encode($data);
  } finally {
    echo 'Done processing data' . PHP_EOL;
  }
};

型
使用$postData = $generator($dataSourceResponse, $apiUrl);调用$generator函数
当我使用print_r作为$postData变量时，浏览器中显示以下内容：Generator Object() Done processing data。
在我看来，从外部API检索的数据没有成功地与$generator函数共享，该函数用于通过API将新闻帖子项目插入Joomla文章。
我在下面提供了完整的PHP脚本供参考。

declare(strict_types=1);
ini_set('error_reporting', E_ALL & ~E_DEPRECATED);

$dataSourceUrl = 'https://api.dtn.com/publishing/news/articles?categoryId=1%2C2%2C3%2C4%2C5%2C6%2C16%2C17&limit=10&maxAge=15&apikey=redacted';
$apiUrl = 'https://upgrade.domain.com/api/index.php/v1';
$token = redacted;
$timeOut = 60;

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => $article->title,
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => $article->topic,
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
    }
    $dataString = json_encode($data);
  } finally {
    echo 'Done processing data' . PHP_EOL;
  }
};

$process = function (string $givenHttpVerb, string $endPoint, string $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

$dataSourceHttpVerb = 'GET';
$dataSourceDataString = '';
$dataSourceHeaders = [
  'Accept: application/json',
  'Accept-Encoding: deflate, gzip, br',
  'Content-Type: application/json',
  'Connection: keep-alive',
];
$dataSourceTimeout = 60;
$dataSourceTransport = curl_init();

try {
  $dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl,   $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);
  $postData = $generator($dataSourceResponse, $apiUrl);

  foreach ($postData as $dataString) {
    if (!is_string($dataString)) {
      continue;
    }
    $curl = curl_init();
    try {
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
  
      $output = $process('POST', $apiUrl, $dataString, $headers, $timeOut, $curl);
  
    } catch (Throwable $e) {
      echo $e->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($curl);
    }
  }
} catch (Throwable $e) {
  echo $e->getMessage() . PHP_EOL;
} finally {
  curl_close($dataSourceTransport);
}

型
下面是基于CLI的脚本的PHP foreach loop的运行情况，我尝试将其改编为参考。

foreach ($articles as $article) {
  $articleData = [
    'id' => 0,
    'catid' => 13,
    'title' => $article->title,
    'introtext' => $article->storySummary,
    'fulltext' => $article->content,
    'note' => $article->topic,
    'state' => 1,
    'access' => 1,
    'created_by' => 386,
    'created_by_alias' => 'DTN News',
    'language' => '*',
  ];

  if (!$articleModel->save($articleData)) {
    throw new Exception($articleModel->getError());
  }
}

型

更新

在数据数组上使用var_dump(json_encode($data)); die();的输出：

{
  "id": 0,
  "catid": 13,
  "title": "DTN Midday Livestock Comments",
  "articletext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "introtext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "fulltext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "note": "DTN\\/Ag\\/Markets",
  "state": 1,
  "access": 1,
  "created_by": 386,
  "created_by_alias": "DTN News",
  "language": "*"
}

型

JSON

1条答案

按热度按时间
utugiqy6

utugiqy61#

我已经通过重构我的代码和实现一些缺失的功能，从我使用的resource作为这个脚本的基础解决了我的问题。我强调了重构的代码和新实现的代码，以及完整的脚本作为参考。
通过将yield (json_encode($data));移动到循环内部来重构$data foreach loop。我还添加了字符串操作函数来清理一些导入的文本内容。

// Strip CSS classes from tags without removing the tags themselves
$stripClasses = '/\bclass=["\'][^"\']*["\']/';

foreach ($resource as $article) {
  $data = [
    'id' => 0,
    'catid' => 13,
    'title' => ucwords(strtolower($article->title)),
    'alias' => strtolower(str_replace(' ', '-', urlencode($article->title))),
    'articletext' => $article->content,
    'introtext' => $article->storySummary,
    'fulltext' => $article->content,
    'note' => 'DTN supplied topics: ' . str_replace('/', ', ', $article->topic),
    'state' => 1,
    'access' => 1,
    'created_by' => 386,
    'created_by_alias' => 'DTN News',
    'language' => '*',
  ];
  
  if (isset($data['articletext']) && !empty($data['articletext'])) {
    $data['articletext'] = strip_tags($data['articletext'], $allowedTags);
    $data['articletext'] = preg_replace($stripClasses, '', $data['articletext']);
  } else {
    $data['articletext'] = '';
  }
  if (isset($data['introtext']) && !empty($data['introtext'])) {
    $data['introtext'] = strip_tags($data['introtext'], $allowedTags);
    $data['introtext'] = preg_replace($stripClasses, '', $data['introtext']);
  } else {
    $data['introtext'] = '';
  }
  if (isset($data['fulltext']) && !empty($data['fullotext'])) {
    $data['fulltext'] = strip_tags($data['fulltext'], $allowedTags);
    $data['fulltext'] = preg_replace($stripClasses, '', $data['fulltext']);
  } else {
    $data['fulltext'] = '';
  }
  
  if ($data === false) {
  } else {
    yield (json_encode(
      $data
    ));
  }
}

字符串
我从资源中包含了这个匿名函数来处理外部和Joomla端点。这对于解决我在重构代码时遇到的404 Resource not found错误至关重要。

$endPoint = function (string $givenBaseUrl, string $givenBasePath, int $givenResourceId = 0): string {
  return $givenResourceId ? sprintf('%s/%s/%s/%d', $givenBaseUrl, $givenBasePath, 'content/articles', $givenResourceId)
: sprintf('%s/%s/%s', $givenBaseUrl, $givenBasePath, 'content/articles');
};

型
我从$postData匿名函数中删除了$apiUrl变量，因为它是不必要的。

$postData = $generator($dataSourceResponse);

型
最后一个try > catch > finally中的第一个foreach loop被重构，以检查API的PATCH或POST，并处理新包含的$endPoint匿名函数。

$storage = [];
foreach ($postData as $dataString) {
  if (!is_string($dataString)) {
    continue;
  }
  $curl = curl_init();
  try {
    $decodedDataString = json_decode($dataString);
    if ($decodedDataString === false) {
      continue;
    }
  
    $headers = [
      'Accept: application/vnd.api+json',
      'Content-Type: application/json',
      'Content-length: ' . mb_strlen($dataString),
      sprintf('X-Joomla-Token: %s', trim($token)),
    ];
    $pk = (int) $decodedDataString->id;
    $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $curl);
  
    $decodedJsonOutput = json_decode($output);
  
    if (isset($decodedJsonOutput->errors)) {
      $storage[] = ['mightExists' => $decodedJsonOutput->errors[0]->code === 400, 'decodedDataString' => $decodedDataString];
      continue;
    }
  
  } catch (Throwable $e) {
    echo $e->getMessage() . PHP_EOL;
    continue;
  } finally {
    curl_close($curl);
  }
}

型
第二个foreach loop被添加来处理错误和检查Joomla文章的重复别名。

foreach ($storage as $item) {
  $storageCurl = curl_init();
  try {
    if ($item['mightExists']) {
      $pk = (int) $item['decodedDataString']->id;
      $item['decodedDataString']->alias = sprintf('%s-%s', $item['decodedDataString']->alias, bin2hex(random_bytes(4)));
      // No need to do another json_encode anymore
      $dataString = json_encode($item['decodedDataString']);
      // HTTP request headers
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-Length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
      $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $storageCurl);
    }
  } catch (Throwable $storageThrowable) {
    echo $storageThrowable->getMessage() . PHP_EOL;
    continue;
  } finally {
    curl_close($storageCurl);
  }
}

型

完整脚本

declare(strict_types=1);
// ini_set('error_reporting', E_ALL & ~E_DEPRECATED);

$dataSourceUrl = Redacted;

$baseUrl= Redacted;
$basePath = 'api/index.php/v1';
$token = Redacted;

$timeOut = 120;

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  $allowedTags = ['p', 'img', 'table'];
  $stripClasses = '/\bclass=["\'][^"\']*["\']/';

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => ucwords(strtolower($article->title)),
        'alias' => strtolower(str_replace(' ', '-', urlencode($article->title))),
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => 'DTN supplied topics: ' . str_replace('/', ', ', $article->topic),
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
  
      if (isset($data['articletext']) && !empty($data['articletext'])) {
        $data['articletext'] = strip_tags($data['articletext'], $allowedTags);
        $data['articletext'] = preg_replace($stripClasses, '', $data['articletext']);
      } else {
        $data['articletext'] = '';
      }
      if (isset($data['introtext']) && !empty($data['introtext'])) {
        $data['introtext'] = strip_tags($data['introtext'], $allowedTags);
        $data['introtext'] = preg_replace($stripClasses, '', $data['introtext']);
      } else {
        $data['introtext'] = '';
      }
      if (isset($data['fulltext']) && !empty($data['fullotext'])) {
        $data['fulltext'] = strip_tags($data['fulltext'], $allowedTags);
        $data['fulltext'] = preg_replace($stripClasses, '', $data['fulltext']);
      } else {
        $data['fulltext'] = '';
      }
  
      if ($data === false) {
      } else {
        yield (json_encode(
          $data
        ));
      }
    }
  } finally {
    // echo 'Done processing data' . PHP_EOL;
  }
};

$endPoint = function (string $givenBaseUrl, string $givenBasePath, int $givenResourceId = 0): string {
  return $givenResourceId ? sprintf('%s/%s/%s/%d', $givenBaseUrl, $givenBasePath, 'content/articles', $givenResourceId)
: sprintf('%s/%s/%s', $givenBaseUrl, $givenBasePath, 'content/articles');
};

$process = function (string $givenHttpVerb, string $endPoint, string   $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

$dataSourceHttpVerb = 'GET';
$dataSourceDataString = '';
$dataSourceHeaders = [
  'Accept: application/json',
  'Accept-Encoding: deflate, gzip, br',
  'Content-Type: application/json',
  'Connection: keep-alive',
];
$dataSourceTimeout = 120;
$dataSourceTransport = curl_init();

try {
  $dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl, $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);
  $postData = $generator($dataSourceResponse);

  $storage = [];
  foreach ($postData as $dataString) {
    if (!is_string($dataString)) {
      continue;
    }
    $curl = curl_init();
    try {
      $decodedDataString = json_decode($dataString);
      if ($decodedDataString === false) {
        continue;
      }
  
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
      $pk = (int) $decodedDataString->id;
      $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $curl);
  
      $decodedJsonOutput = json_decode($output);
  
      if (isset($decodedJsonOutput->errors)) {
        $storage[] = ['mightExists' => $decodedJsonOutput->errors[0]->code === 400, 'decodedDataString' => $decodedDataString];
        continue;
      }
  
    } catch (Throwable $e) {
      echo $e->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($curl);
    }
  }

  foreach ($storage as $item) {
    $storageCurl = curl_init();
    try {
      if ($item['mightExists']) {
        $pk = (int) $item['decodedDataString']->id;
        $item['decodedDataString']->alias = sprintf('%s-%s', $item['decodedDataString']->alias, bin2hex(random_bytes(4)));
        // No need to do another json_encode anymore
        $dataString = json_encode($item['decodedDataString']);
        // HTTP request headers
        $headers = [
          'Accept: application/vnd.api+json',
          'Content-Type: application/json',
          'Content-Length: ' . mb_strlen($dataString),
          sprintf('X-Joomla-Token: %s', trim($token)),
        ];
        $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $storageCurl);
      }
    } catch (Throwable $storageThrowable) {
      echo $storageThrowable->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($storageCurl);
    }
  }
} catch (Throwable $e) {
  echo $e->getMessage() . PHP_EOL;
} finally {
  curl_close($dataSourceTransport);
}

型

赞(0）分享  回复(0）举报  12个月前
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com