主页 / 问题

问题[парсер]

Martin Hope
Nik
Asked: 2024-11-06 15:55:16 +0000 UTC
  • 5

由于某种原因，我的查询数据没有被正确解析。

为了不拖一堆代码，我删除了所有不必要的东西，只留下有问题的部分本身。

我有一个简单的获取请求

http://localhost:3002/test?filter[15][]=1

并且有一条处理它的路线

  @Get('test')
  test(@Query() query) {
    return query;
  }

没有更多的代码了。

出于某种原因，当被问到时

http://localhost:3002/test?filter[15][]=1

值15丢失，返回结果

{ filter: [ [ '1' ] ] }

如果您在过滤器括号中指定大于20 的值，例如

http://localhost:3002/test?filter[21][]=1

然后返回正确的值

{ filter: { '21': [ '1' ] } }

如果有人可以，请告诉我为什么会发生这种情况，以及如何纠正这种情况，以便查询始终具有与第二个结果相同的输出。谢谢。

парсер
Martin Hope
Adam
Asked: 2024-10-09 18:09:13 +0000 UTC
  • 5

请帮我解决问题。我从网站上抓取价格。示例页面。 Price 有一个 XPath

/html/body/div[8]/div/div[2]/div[1]/div[2]/div/div[2]/div[2]/div[2]/div[2]/div[1]/div/span[2]/span[2]

但在询问解析器后我得到一个空值。类似的路径适用于其他站点。我安装了 XPath 帮助器插件，它产生以下结果

/html[@class='bx-core bx-win bx-no-touch bx-no-retina bx-chrome desktop portrait']/body[@class='auth_no widget-on b24-on']/div[@class='main_wrap']/div[@class='shadow_wraper']/div[@class='content_wrap']/div[@class='content_right_wrap goods']/div[@id='bx_117848907_309101']/div/div[@class='catalog_item_wrap']/div[@class='catalog_item sw']/div[@class='catalog_item_right']/div[@class='col-lg-9 col-sx-9 col_mob-full']/div[@class='catalog_price_wrap row sw']/div[@class='catalog_price_desc_wrap col-lg-12 col-xs-12']/span[@class='price']/span[@id='bx_117848907_309101_price']

如果你只是复制元素，它看起来像这样：

<span class="price_num xh-highlight" id="bx_117848907_309101_price" data-entity="final-price" data-price="2262" data-currency="RUB" style="font-size: 18px;">2 262 Р</span>

我做错了什么？我应该给解析器提供哪条路径？

PS我很抱歉，我不是程序员，但需要找到解决这个问题的方法。我使用batchurlscraper

парсер
Martin Hope
Артём Ионаш
Asked: 2024-07-19 18:06:14 +0000 UTC
  • 5

/train.zip例如，如果我拥有PNG 图像的存档及其/metadata.csv注释文件，以便parquet-converter机器人可以自动识别并正确解释该数据集，那么我应该在 Hugging Face Hub 平台中使用什么文件结构？

期望结果的示例

但无论我使用什么文件排列方式，

/train.zip
/metadata.csv

或者

/train/train.zip
/metadata.csv

我得到一个例外：

Cannot load the dataset split (in streaming mode) to extract the first rows.
Error code:   StreamingRowsError
Exception:    ValueError
Message:      One or several metadata.csv were found, but not in the same directory or in a parent directory of zip://1.png::hf://datasets/[user]/[repo-name]@[hash]/train/train.zip.
Traceback:    Traceback (most recent call last):
                File "/src/services/worker/src/worker/job_runners/split/first_rows.py", line 322, in compute
                  compute_first_rows_from_parquet_response(
                File "/src/services/worker/src/worker/job_runners/split/first_rows.py", line 88, in compute_first_rows_from_parquet_response
                  rows_index = indexer.get_rows_index(
                File "/src/libs/libcommon/src/libcommon/parquet_utils.py", line 640, in get_rows_index
                  return RowsIndex(
                File "/src/libs/libcommon/src/libcommon/parquet_utils.py", line 521, in __init__
                  self.parquet_index = self._init_parquet_index(
                File "/src/libs/libcommon/src/libcommon/parquet_utils.py", line 538, in _init_parquet_index
                  response = get_previous_step_or_raise(
                File "/src/libs/libcommon/src/libcommon/simple_cache.py", line 590, in get_previous_step_or_raise
                  raise CachedArtifactError(
              libcommon.simple_cache.CachedArtifactError: The previous step failed.
              
              During handling of the above exception, another exception occurred:
              
              Traceback (most recent call last):
                File "/src/services/worker/src/worker/utils.py", line 96, in get_rows_or_raise
                  return get_rows(
                File "/src/libs/libcommon/src/libcommon/utils.py", line 197, in decorator
                  return func(*args, **kwargs)
                File "/src/services/worker/src/worker/utils.py", line 73, in get_rows
                  rows_plus_one = list(itertools.islice(ds, rows_max_number + 1))
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 1389, in __iter__
                  for key, example in ex_iterable:
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 234, in __iter__
                  yield from self.generate_examples_fn(**self.kwargs)
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/packaged_modules/folder_based_builder/folder_based_builder.py", line 376, in _generate_examples
                  raise ValueError(
              ValueError: One or several metadata.csv were found, but not in the same directory or in a parent directory of zip://1.png::hf://datasets/[user]/[repo-name]@[hash]/train/train.zip.

我究竟做错了什么？

парсер
Martin Hope
Pureya
Asked: 2023-09-22 13:31:28 +0000 UTC
  • 5

您需要先显示商品的最低价格

  import requests
  from bs4 import BeautifulSoup


  for i in range(1, 6):
      print(f"Parsing {i} page")
      url = "https://cvetland.kz/almaty/product-category/catalog/?utm_term=%D0%B4%D0%BE%D1%81%D1%82%D0%B0%D0%B2%D0%BA%D0%B0%20%D1%86%D0%B2%D0%B5%D1%82%D0%BE%D0%B2%20%D0%BD%D0%B0%20%D0%B4%D0%BE%D0%BC%20%D0%BA%D1%80%D1%83%D0%B3%D0%BB%D0%BE%D1%81%D1%83%D1%82%D0%BE%D1%87%D0%BD%D0%BE&matchtype=b&utm_source=google&utm_medium=cpc&utm_campaign=!Zakaz_zvetov_|_Poisk_|_Almaty&gad=1&gclid=EAIaIQobChMIuuHfz7CvgQMVWQkGAB0OaASIEAAYAiAAEgI1GfD_BwE{i}/"
      response = requests.get(url)
      soup = BeautifulSoup(response.text, 'lxml')

      names = soup.find_all("a", class_ = "question__title-link question__title-link_list")
      prices = soup.find_all("span", class_ = "woocommerce-Price-amount amount")


      for j in range(0, len(names)):
          print(names[j].text)
      for j in range(0, len(prices)):
          print(prices[j].text)
парсер
Martin Hope
shezyy
Asked: 2023-04-20 04:39:43 +0000 UTC
  • 7

我正在编写一个应该采用折扣结束日期的解析器，但出于某种原因，采用了网站的英文版本，尽管在标头中指定了接受语言。请帮忙解析俄罗斯数据！

import requests
from bs4 import BeautifulSoup
import re


def get_data(url):
    headers = {
        'Accept-Language': 'ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7',
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) '
                      'Chrome/112.0.0.0 Safari/537.36'

    }

    req = requests.get(url, headers)
    soup = BeautifulSoup(req.text, 'lxml')
    date = soup.find('p', class_='game_purchase_discount_countdown').text
    regex_pattern = r'(\d{1,2})\s+(\w+)'

    match = re.search(regex_pattern, date)

    if match:
        day = match.group(1)
        month = match.group(2).capitalize()
        print(f"{day} {month}")
    

get_data("https://store.steampowered.com/app/552500/Warhammer_Vermintide_2/")

结论：

25 April
парсер