• Nighed@feddit.uk
    link
    fedilink
    English
    arrow-up
    6
    ·
    edit-2
    1 day ago

    How much does it spam, if I run it am I likely to get ip banned anywhere?

    …Or in trouble for ‘visiting’ unsavoury sites?

    • kabi@lemm.ee
      link
      fedilink
      arrow-up
      5
      ·
      23 hours ago

      There are always several projects to choose from.

      The URLs project plays it fast and loose and archives an assortment of random URLs. This one has an IP block warning.

      Some have NSFW warnings.

      Other projects aim to archive a single site as accurately as possible (possibly with a deadline when the site is shutting down), so they can’t afford to have their warriors blocked or rate limited. If you are, that would be because of an issue. You can choose to archive sites you don’t want to visit to avoid issues.

      • Nighed@feddit.uk
        link
        fedilink
        English
        arrow-up
        5
        ·
        23 hours ago

        Ah, I thought it was just a monolithic app you set going and have no control over. Ty

        • kabi@lemm.ee
          link
          fedilink
          arrow-up
          4
          ·
          23 hours ago

          np. Right now you could set it to ask.fm (shuts down dec 1.). Zero IP blocking or rate limiting. Puts your machine to good use.

          Not all projects listed in the warrior are actually active. Check out https://tracker.archiveteam.org/ for all the current projects and see if the one you want to archive is actually active (has people receiving and sending in items)

    • Appoxo@lemmy.dbzer0.com
      link
      fedilink
      arrow-up
      2
      ·
      edit-2
      24 hours ago

      Just limit it to one job per session.
      I ran 3 continuous jobs while archiving reddit and could still connect without issue.