Commit Graph

2486 Commits

Author SHA1 Message Date
Dag 41df17bc46
refactor (#3712)
* test: refactor test suite

* docs

* refactor

* yup

* docs
2023-10-01 19:23:30 +02:00
sysadminstory 0c92cf32d4
[ImgsedBridge] Fix and improvements (#3710)
* [ImgsedBridge] Fix and improvements

- Display an error if the user doesn't select at least an content type
  to display
- Unsplit the regular expression to make the URL of imgsed.com work too
- Remove the "hour part" of the publication date : the website shows
  only the number of days if the content is older than one day

* [ImgsedBridge] Fix and improvements

Fix syntax

* [ImgsedBridge] Fix and improvements

- Fix TEST_DETECT_PARAMETERS
- change detectParameters regular expression to match more instagram.com
  URLs

* [ImgsedBridge] Fix and improvements

- Fix date parsing for interval 'a day'

* lint

---------

Co-authored-by: Dag <me@dvikan.no>
2023-10-01 19:00:13 +02:00
Dag 7273a05f02
fix: google play and tiktok (#3722)
* fix(googleplay)

* fix(tiktok)
2023-10-01 18:53:50 +02:00
ORelio 3557e5ffd4
[CssSelector/Sitemap] Minor fixes (#3719)
- Apply title_cleanup to title from metadata (#3717)
- Metadata: Fix ld+json object/array confusion
- Sitemap: Also try /sitemap.xml well known url
2023-09-30 15:03:52 +02:00
Dag 2172df9fa2
fix: various notice fixes (#3718) 2023-09-29 19:17:03 +02:00
Dag b9ec6a0eb4
feat: add manyvids bridge (#3716) 2023-09-29 00:39:24 +02:00
Dag ae53adefad
refactor: FeedItem::setTimestamp() (#3711) 2023-09-26 00:27:45 +02:00
Dag f421c45b21
test: add feed item test (#3709)
* test: add feed item test

also some refactor

* yup

* yup
2023-09-25 22:32:15 +02:00
ORelio e1b911fc1f
[CssSelectorBridge] Retrieve metadata for social media embeds (#3602, #3687) (#3706)
* [CssSelectorBridge] Metadata from social embed (#3602, #3687)

Implement the following metadata sources:
 - Facebook Open Graph
 - Twitter <meta> tags
 - Standard <meta> tags
 - JSON linked data (ld+json)

The following metadata is supported:
 - Canonical URL (may help removing garbage from URLs)
 - Article title
 - Truncated summary
 - Published/Updated timestamp
 - Enclosure/Thumbnail image
 - Author Name or Twitter handle

SitemapBridge will also automatically benefit from this commit.

* [php8backports] Add array_is_list()

Needed this function for ld+json implementation in CssSelectorBridge.

* [SitemapBridge] Add option to discard thumbnail

* [CssSelectorBridge] Fix linting issues
2023-09-24 23:07:43 +02:00
Dag f321f000c1
feat: add url component (#3684)
* feat: add url library

* fix
2023-09-24 18:34:09 +02:00
Dag 437afd67e0
fix: various fixes (#3702)
* fix: symfonycasts

* various fixes
2023-09-24 18:15:14 +02:00
ORelio ce353c1e4f
[CssSelectorBridge] Fix URL filtering (#3676) (#3701)
Co-authored-by: tougaj <tougaj@users.noreply.github.com>
2023-09-24 16:12:30 +02:00
Dag 0dc6c66840
fix: add duration (#3699) 2023-09-24 00:03:21 +02:00
Dag d33808ea9e
fix: image (#3698) 2023-09-23 23:49:01 +02:00
Dag 0c69148cff
fix(vice): news rss changed (#3694)
* fix: typo in prior commit

* fix(vice): news rss changed
2023-09-23 20:39:02 +02:00
Dag bab02bf190
fix(flickr) (#3692) 2023-09-23 19:29:04 +02:00
Dag cb6c931b1f
fix(duckduckgo): order by date (#3689) 2023-09-23 17:50:41 +02:00
Dag a6a1d553d9
tweaks (#3686) 2023-09-22 20:59:45 +02:00
Dag 39d6710798
fix(twitch) (#3685) 2023-09-22 20:41:39 +02:00
mruac a3c29f3a52
resolve comment (#3683)
https://github.com/RSS-Bridge/rss-bridge/pull/3617#issuecomment-1730244049
2023-09-22 09:38:05 +02:00
User123698745 7a9bfa1087
[YoutubeBridge] handle new youtube description system / fix missing description (#3682)
* [YoutubeBridge] handle new youtube description system

* [YoutubeBridge] fix unrelated warnings

* [YoutubeBridge] discard everything when one link can not be matched & add more boundary chars

* [YoutubeBridge] rebase on master & minor fixes
2023-09-22 05:40:13 +02:00
Dag 7329b83cc0
refactor: logger (#3678) 2023-09-21 22:05:55 +02:00
Julien Papasian 360f953be8
Fix #3674 - [Nautiljon] Remove requirement of cURL NSS (#3679)
Looks like it works with OpenSSL now.
2023-09-21 18:55:53 +02:00
Dag 0bf38e5c56
fix: small notice errors (#3677)
* fix notice

* fix notice

* tweak

* tweaks
2023-09-20 03:15:15 +02:00
Scott Colby cf7e3eea56
Add DeutscheWelle FeedExpander bridge. (#3673)
* [DeutscheWelle] Add DeutscheWelle FeedExpander bridge.

* [DeutscheWelle] Fix linting errors.
2023-09-15 23:41:08 +02:00
User123698745 3b91b1d260
[XPathBridge] add option to skip htmlspecialchars (#3672) 2023-09-15 01:58:06 +02:00
Dag 409236e48e
fix: logic bug in 429 caching logic (#3669) 2023-09-14 03:26:01 +02:00
Dag bb7f329e81
fix(instructables): migrate from dom to json api (#3667) 2023-09-13 22:48:37 +02:00
mruac 3e1e96e477
[PatreonBridge] resolve null coalescing issue (#3664)
* extend post presentation

* applied phpcbf

note: phpcs does not like long null coalescing chains

* resolved phpcs

* resolved github comment https://github.com/RSS-Bridge/rss-bridge/pull/3617/#issuecomment-1699568400

* .

* lint SteamAppNewsBridge
2023-09-11 13:15:14 +02:00
ImportTaste a9cf1512e7
[SteamAppNewsBridge] Add tags filter (#3662)
Undocumented tags filter discovered through /ISteamWebAPIUtil/GetSupportedAPIList/v1/
e.g. /ISteamNews/GetNewsForApp/v2/?appid=1091500&tags=patchnotes
2023-09-11 03:35:09 +02:00
Dag 3178deb5a8
fix: mastodon, cache tweaks, docs (#3661)
* cache tweaks

* docs

* fix(mastodon): type bug
2023-09-10 23:35:40 +02:00
Dag 4b9f6f7e53
fix: rewrite and improve caching (#3594) 2023-09-10 21:50:15 +02:00
July 586d707ae4
[ArsTechnicaBridge] Add new bridge (#3657) 2023-09-09 09:19:09 +02:00
mruac b3a7842448
[PixivBridge] Add cookie auth and options (#3653)
* added cookie mgmt and support for issue
https://github.com/RSS-Bridge/rss-bridge/issues/2759

* added image proxy option

* + mature and ai options, + cookie doc

* mention doc

* check cookie is auth'd
2023-09-06 16:16:25 +02:00
csisoap dbe37cc302
[TwitterBridge] Filter out any promoted tweet (#3652)
* Filter out any advertise tweet

* Make some filter work, fix bug that may happen with tweet id list.

* clear phpcs warning, ignore line length warning
2023-09-06 16:14:11 +02:00
sysadminstory 38b957398a
[AutoJMBridge] Fix content extraction (#3649)
* [AutoJMBridge] Fix content extraction

- Website changed, bridge was updated accordingly
- Added the function detectParameters
- Added the test array for the detectParameters function

* [AutoJMBridge] Fix test

Fix content of the TEST_DETECT_PARAMETERS array

* [AutoJMBridge] Update exaù^me value parameter

Example value was not valid anymore, so it was updated
2023-09-05 02:12:47 +02:00
User123698745 99b86c0e1c
[GithubSearchBridge] repair bridge / handle new search ui (#3647) 2023-09-04 03:00:08 +02:00
Niehztog 92b2bc5e11
fixes extracting article images, article date/time and article author and item id (#3645) 2023-09-03 00:22:48 +02:00
R3dError 64000a2526
[NACSouthGermanyMediaLibraryBridge] Add new bridge (#3636)
* Init nac south bridge

* Rename bridge

* Refactoring

* Refactor

* Fix formatting

* Fix testing errors

* Change constants

* Update logo

* Remove author omission in descriptions

* Fix comment

* Add maintainer

* Rename bridge

* Add technical note to bridge description
2023-08-30 19:01:55 +02:00
csisoap 4d05d0beff
[TwitterBridge] Add support for OAuth authorization. (#3628)
* Update TwitterClient.php

- Add OAuth authorization header.
- Add new endpoint.

* Update TwitterBridge.php

- Make some changes to support new endpoint.

* Update TwitterBridge.php

* clean up, fix warning

* fix warning

* fix warning

* remove oauth token

* fix wrong twitter id when encounter reply post.

* Update TwitterClient.php

* fix wrong twitter id cause by previous commit

* clear warning

* attempt to clear warning

* attempt to clear warning
2023-08-29 17:14:34 +02:00
mruac 9707586ee8
[PatreonBridge] Extend the presentation of parsed posts (#3617)
* extend post presentation

* applied phpcbf

note: phpcs does not like long null coalescing chains

* resolved phpcs
2023-08-29 17:09:05 +02:00
mruac f0ec797f4b
[FurAffinityBridge] Option for instance host to add custom cookie (#3638)
* added custom cookie config

* appease phpunit
2023-08-29 17:05:37 +02:00
mruac 9e33a15b93
[FurAffinityBridge] Fix if search result contains hidden submission (#3637)
* Reverts to preview submission if full is hidden

* Reverts to preview submission if full is hidden

* revert

* added fallback to higher res preview if SWF

* amend
2023-08-29 17:04:10 +02:00
Paul 00a18a1cd1
Merge pull request #3639 from Mynacol/gitlab-epic
[GitlabIssueBridge] Add support for GitLab Epics
2023-08-27 13:32:04 +02:00
Mynacol 14607c07f6 [GitlabIssueBridge] Fix example values for MR
These values are used for testing and PR artifacts, but
https://gitlab.com/fdroid/fdroidclient currently has no MR !2099,
leading to a HTTP 404 error. This just uses issue #1 and MR !1.

To support epics, the specified repository is ignored.
2023-08-27 13:27:37 +02:00
Mynacol 999d5dce40 [HeiseBridge] Remove archive link for heise+
archive.ph is also not able to provide the full content of paywalled
heise+ articles.
2023-08-27 13:01:09 +02:00
Mynacol c3b5b382ba [ZeitBridge] Remove broken paywall workaround
Clean up spoofing Google Bot as this workaround doesn't work anymore.
2023-08-27 12:57:36 +02:00
Mynacol 18a8a51271 [GitlabIssueBridge] Add support for GitLab Epics 2023-08-27 12:13:44 +02:00
t0stiman 0325c2414a
fix carthrottlebridge (#3633) 2023-08-25 12:34:35 +02:00
Lars Stegman eb4ff7099f
CSS Selector Bridge 2 (#3626)
* [CssSelector2Bridge] Implement CSS Selector bridge 2

* [CssSelector2Bridge] Fix author not being loaded

* [CssSelector2Bridge] Remove unneeded time nullcheck

* Fix linting

* Fix failing test

* Implement PR fixes

* Update bridges/CssSelector2Bridge.php

Co-authored-by: ORelio <ORelio@users.noreply.github.com>

* Rename bridge and fix syntax error for php7

---------

Co-authored-by: ORelio <ORelio@users.noreply.github.com>
2023-08-22 21:28:16 +02:00
Dag 54045be951
fix(tpb): add missing cat (#3631) 2023-08-22 20:06:16 +02:00
Dag 3ac861a866
fix(twitch): Invalid argument supplied for foreach() at bridges/TwitchBridge.php line 115 (#3630) 2023-08-22 19:47:32 +02:00
veloute c5cbab1231
Update TheGuardianBridge.php (#3629) 2023-08-22 17:59:06 +02:00
Eugene Molotov 959dd937b4
[VkBridge] Using more universal regular expression to generate item title (#3627) 2023-08-21 04:53:54 +02:00
John S Long 79e3f7f204
[MastodonBridge] Add support for excluding regular statuses (non-boosts/replies) (#3624)
* [MastodonBridge]: add support for excluding posts (non-boosts/replies)

* Update name of input

* Fix lint failures
2023-08-20 04:37:21 +02:00
Corentin Garcia a1237d90f1
[RainbowSixSiegeBridge] fix links, date and img tag (#3619) 2023-08-15 16:16:06 +02:00
mruac 28077155ca
[PatreonBridge] Resolve creator name in feed name (#3616)
* resolve creators without custom url

* hint how to enter creator with non-custom url

* applied phpcbf
2023-08-15 16:04:09 +02:00
mruac 7a1180c80f
bridges: added Itaku.ee Bridge (#3615)
* Fix php8.2 deprecated warning

Fix php8.2 warning: `Deprecated: Creation of dynamic property is deprecated`

* modified tag presentation

* renamed to fit naming convention

* undo commit

* applied phpcbf and phpunit
2023-08-15 16:02:58 +02:00
Dag ce72503df6
fix(codeberg): change dom selectors for timestamp and content, fix #3610 (#3611) 2023-08-11 19:02:38 +02:00
George Sokianos d55994643d
bridges: Added Ko-Fi.com bridge (#3609)
* Added Ko-Fi.com bridge

* Changed the exampleValue based on KoFiBridge-pr-context1 artifacts

* Fixed "Undefined array key 0" error

* fixed PHPCS issues

---------

Co-authored-by: George Sokianos <George.Sokianos@hostelworld.com>
2023-08-11 16:16:53 +02:00
Christian Schabesberger 11ea6aedfd
hide dpa articles in Nordbayern News (#3608) 2023-08-10 23:59:37 +02:00
sysadminstory 52d3cce59d
bridges: add context to detectParameters (#3607)
* bridges: add context to detectParameters

Some bridges did not return the context parameter but they used it in
the parameters

* bridges: add context to detectParameters

Fix test for InstagramBridge
2023-08-09 22:40:24 +02:00
ORelio 6cc4cf24dc
[FuturaSciences] Fix content extraction (#3487, #3488) (#3606) 2023-08-09 20:10:15 +02:00
sysadminstory 1fcf67f14a
[PepperBridgeAbstract] Fix deal origin (#3605)
Origin display has chenged : this commit follow the websites changes.

Fixes #3521
2023-08-09 17:36:02 +02:00
sysadminstory f3896ed543
[ImgsedBridge] Add detectParameters feature to the bridge (#3604)
The bridge can detect the most common profile variation URL of
instagram.com or imgsed.com websites to extract the username.
2023-08-09 17:35:35 +02:00
ORelio b86ee5778b
[SitemapBridge] Add SitemapBridge (#3602)
* [SitemapBridge] Add SitemapBridge

This bridge is a variant of CssSelectorBridge.
Instead of retrieving article list from home page,
retrieves article list from SEO sitemap.xml.
Requires CssSelectorBridge to be installed.

* [SitemapBridge] Code linting
2023-08-08 15:02:01 +02:00
adminvulcano 43ec82179b
[TldrTechBridge] Add Cybersecurity section (#3601) 2023-08-08 07:00:07 +02:00
Korytov Pavel cf6d94dc2a
[EconomistBridge] Fix strange image urls (#3600) 2023-08-08 06:58:08 +02:00
Niehztog 3e3481bd7a
adds Nius bridge (#3599)
* adds Nius bridge

* fix linter errors

* fix linter errors

* fix linter errors

* fix extract author
2023-08-07 05:33:35 +02:00
User123698745 4976cd227e
[FeedExpander] support xhtml content / content with child elements (#3598)
* [core] support xhtml content type in FeedExpander

* [FilterBridge] change defaultValue to exampleValue

* [core] support content with child elements in FeedExpander
2023-08-04 22:14:08 +02:00
Tone d32419ffcf
added the option for a sessioncookie in heiseBridge (#3596)
* added the option for a sessioncookie

with a valid cookie you can get full heise+ (paywall) articles

* formating

* lint

---------

Co-authored-by: Dag <me@dvikan.no>
2023-08-03 22:43:55 +02:00
Dag 7e4807530e
fix: various small fixes (#3580) 2023-07-31 20:43:18 +02:00
Dag 8b6eecea25
docs: add note about expensive operation (#3579) 2023-07-31 20:43:11 +02:00
ORelio f8fd05f08f
[CssSelectorBridge] Handling of missing links (#3585)
When using parent element as URL selector:

* If no <a> inside some elements, ignore them
* If no <a> inside ALL elements, report an error

Fixes #3573 #issuecomment-1656943318
2023-07-31 19:07:34 +02:00
User123698745 f957eea300
[FallGuysBridge] new bridge (#3584) 2023-07-31 01:05:38 +02:00
mrtnvgr 3a57fc800b
DoujinStyleBridge: Update html tags (#3581) 2023-07-30 06:46:16 +02:00
Dag 701fe3cfed
fix: various small fixes (#3578) 2023-07-29 00:14:30 +02:00
Aaron F 11ce8b5dcd
CVEDetails got a new HTML layout. (#3577)
This fixes the parser for CVEDetails.
2023-07-27 23:54:17 +02:00
Korytov Pavel f5f76f111b
[TldrTechBridge] Add Web Dev and Founders sections (#3576) 2023-07-26 22:59:49 +02:00
Korytov Pavel bf4ea12719
[ScientificAmerican] Fix bridge (#3575) 2023-07-26 21:47:47 +02:00
ORelio 235c084820
[DilbertBridge] Remove bridge (#3574)
dilbert.com has closed down.
2023-07-26 20:41:48 +02:00
ORelio 977c0db382
[CssSelectorBridge] Improvements (#3537) (#3573)
* [CssSelectorBridge] Improvements (#3537)

* Improve parameter documentation / add tooltips
* Allow extracting content from home page instead of article page
* Keep titles from home page when every page <title> is the same

* [CssSelectorBridge] Code linting

* [CssSelectorBridge] Code linting (2)

* [CssSelectorBridge] Code linting (3)
2023-07-26 19:41:29 +02:00
csisoap 556bca58cf
[TwitterBridge] Fix search, user, list ID (#3566)
* Add ability to fetch user, list tweet

* Fix user, search, list ID although list still broke

* clear whitespace

* Revert CACHE_TIMEOUT

* clear whitespace, change single quote

* Clear PHP warning, add ability to get full-text if truncated

* Clear PHP warning

* clear warning

* clear whitespace

* Add check condition for mediaDetails.

* Add whitespace

* Add try catch exception for get full-text tweet

* clear warning

* clear warning
2023-07-25 22:36:41 +02:00
Dawid Wróbel 2cc89b767c
[AllegroBridge] fix non-functional bridge (#3571)
— fix cookie pattern
– use data analytics attributes wherever possible to avoid relying on obfuscated class names
— add support for promoted offers
— include sponsored and promoted offers by default
— some additional refactoring
2023-07-25 20:52:47 +02:00
Simon Alberny 1f6c2cd32c
Allocine Sorties movie date added (#3569) 2023-07-24 16:25:09 +02:00
Dag 74635fd752
fix(DisplayAction): improve error handling and cache logic (#3558)
* fix(DisplayAction): improve error handling and cache logic

* restore prev timeouts

* refactor

* yup

* test: fix unit test

* leave twitter client unchanged

* leave twitter bridge unchanged
2023-07-23 23:05:35 +02:00
Eugene Molotov 38ca124de0
[VkBridge] Better title generation (#3563)
1. Use first parargraph only
2. Remove tags
3. Allow to use comma and colon in title
2023-07-22 14:00:12 +02:00
Dag 39a8346c53
fix(pokemonnews): throw if antibot, #3327 (#3562) 2023-07-21 20:52:20 +02:00
Dag d08b2616ef
feat(twitter): use account icon as feed icon, fix #3348 (#3561) 2023-07-21 20:26:22 +02:00
Predä 663729cf19
[TikTokBridge] Use another way to get videos infos to include video link (#3557)
* [TikTokBridge] Use another way to get videos infos to include video link

* [TikTokBridge] Use cover if dynamicCover is empty

* [TikTokBridge] Add support for the rest of item params
2023-07-20 05:50:45 +02:00
Predä 2ffb54c7c2
[PicukiBridge] Add count parameter (#3556) 2023-07-20 00:52:09 +02:00
Dag a4a328583a
fix(reddit): set custom http ua to fix 429 errors (#3552)
* refactor

* refactor

* fix(reddit): set custom http ua to fix 429 errors

* lint
2023-07-19 06:39:17 +02:00
Dag 6254b8593e
refactor(cache): extract and encapsulate cache expiration logic (#3547)
* refactor(cache): extract and encapsulate cache expiration logic

* fix: logic bug in getSimpleHTMLDOMCached

* fix: silly me, index should of course be on the key column

* silly me again, PRIMARY keys get index by default lol

* comment out the delete portion in loadData

* remove a few log statements

* tweak twitter cache timeout
2023-07-19 05:05:49 +02:00
sysadminstory 087e790ec1
[ImgsedBridge] Add new Instagram Bridge Alternative (#3550)
* [ImgsedBridge] Add new Instagram Bridge Alternative

Imgsed is a Website adverstised on instagram website, that's is not
behind Cloudflare Anti Bot feature.
You can select to display Posts, Tags, and Stories of a specific
username

* [ImgsedBridge] Fix empty defaultValue
2023-07-19 03:28:14 +02:00
mrtnvgr 4ce63c88aa
Add DoujinStyleBridge (#3549)
* v1

* improve title

* search support

* random support

* fix categories

* add metadata to content

* fix linter errors

* i'm sorry
2023-07-18 20:48:29 +02:00
Paroleen a1bae7a9a8
[SpotifyBridge] Add search API support (#3548) 2023-07-18 00:43:08 +02:00
Dag 440adf2f3b
fix(githubissue): add 10 min cache (#3545) 2023-07-16 22:28:20 +02:00
Dag a59793e8d6
refactor: extract CurlHttpClient (#3532)
* refactor: extract CurlHttpClient

* refactor

* interface
2023-07-16 22:07:34 +02:00
Dag 7b46b97abd
refactor(spotify): replace manual curl with getContents (#3544) 2023-07-16 21:50:44 +02:00
Thomas e5729fdaac
[YouTubeCommunityTabBridge] Fix PHP warnings for posts w/o text (#3536)
Because "contentText" is always present, PHP warnings were previously
generated for posts without text.
Existence of sub-property "runs" gets checked now to avoid this.
2023-07-15 15:18:09 +02:00