Commit Graph

77 Commits

Author SHA1 Message Date
Dag 4c3ebb312d
feat: improve error handling ux (#3298)
* feat: improve error handling ux

* feat: add error messages for failed xml parsing
2023-03-20 19:11:51 +01:00
Mynacol 4450e9b973
Let curl select the default HTTP version (#3249)
This essentially reverts b042412416,
as YouTube seems to have fixed their servers.
At least I was able to query the YouTube endpoint around 150 times with
CURL_HTTP_VERSION_2TLS recently. They even advertise HTTP/3 support with
an `alt-svc` HTTP header now.

This unsets CURLOPT_HTTP_VERSION to let curl decide
on the version. This would support all curl versions and opens the
possibility for HTTP/3, but leads to inconsistent behavior depending
on the underlying curl version.

We don't set CURL_HTTP_VERSION_NONE explicitly, as it is always the curl
default and opens the path to let individual bridges override the HTTP
version where necessary.

Alternatively, setting CURL_HTTP_VERSION_2TLS explicitly would lead to consistent behavior
regardless of the curl version, but might uncover old curl bugs before the
developers enabled HTTP/2 by default.
Additionally, that requires at least PHP 7.0.7 (we require PHP 7.4
already) and curl 7.47.0 [1], released on Jan 27 2016 [2].

See also the discussion on https://github.com/RSS-Bridge/rss-bridge/pull/3249

[1] https://www.php.net/manual/curl.constants.php
[2] https://curl.se/docs/releases.html
2023-02-22 17:48:39 +01:00
Dag a13c4624fb
feat: custom http ua in AO3, fix #3188 (#3189)
* refactor

* feat: custom http ua in AO3, #3188
2022-12-13 09:53:42 +01:00
Dag a5779d30b5
feat: add max file size to http responses (#3140) 2022-11-16 17:56:26 +01:00
Dag 2ef98b299f
refactor: extract frontpage to template (#3130)
Also introduce usage of Response object
2022-11-07 18:22:54 +01:00
Dag e027bd9274
fix: improve FeedExpander (#3103)
* fix: improve FeedExpander

Include the first libxml error in exception.

Give better error message if trying to parse the empty string.

Log all libxml errors if debug mode is enabled.

* error handling and logging tweak
2022-10-29 10:27:02 +02:00
Dag 1f576312ea
feat: in debug mode, include part of http response in exception message (#3090) 2022-10-29 08:42:50 +02:00
Dag 27b3d7c34e
feat: improve logging and error handling (#2994)
* feat: improve logging and error handling

* trim absolute path from file name

* fix: suppress php errors from xml parsing

* fix: respect the error reporting level in the custom error handler

* feat: dont log error which is produced by bots

* ignore error about invalid bridge name

* upgrade bridge exception from warning to error

* remove remnants of using phps builin error handler

* move responsibility of printing php error from logger to error handler

* feat: include url in log record context

* fix: always include url in log record contect

Also ignore more non-interesting exceptions.

* more verbose httpexception

* fix

* fix
2022-09-08 19:07:57 +02:00
Eugene Molotov bbbd599bc8
fix: do not throw exception on 301-303 http responses (#3014) 2022-09-06 15:40:20 +02:00
Dag 2bbce8ebef
refactor: general code base refactor (#2950)
* refactor

* fix: bug in previous refactor

* chore: exclude phpcompat sniff due to bug in phpcompat

* fix: do not leak absolute paths

* refactor/fix: batch extensions checking, fix DOS issue
2022-08-06 22:46:28 +02:00
Dag b042412416
fix: force HTTP 1.1 in curl requests (#2949)
Since curl 7.62.0 the default option is: CURL_HTTP_VERSION_2TLS

Before that the default used to be: CURL_HTTP_VERSION_1_1

Fix #2947
2022-08-05 11:46:32 +02:00
Dag afcc38786e
fix: use default headers in getContents() (#2927) 2022-07-31 04:21:56 +02:00
Jan Tojnar 5b5f3b4254
Do not use constants for configuration (#2938)
* docs: Do not use constant names when referring to config options

The options are customizable using a config file and no longer hardcoded in index.php since 8ac8e08abf

* Do not use constants for configuration

Since <8ac8e08abf>, they are just set to the configuration object values.
2022-07-24 19:26:12 +02:00
Jan Tojnar d107f8ed30
Improve Factory variable names (#2895) 2022-07-06 12:14:04 +02:00
Dag 321ec7c8c1
refactor: move cache logic into the factory (#2884) 2022-07-05 13:20:01 +02:00
Dag 4f75591060
Reformat codebase v4 (#2872)
Reformat code base to PSR12

Co-authored-by: rssbridge <noreply@github.com>
2022-07-01 15:10:30 +02:00
Dag 07927008eb
refactor: CacheFactory (#2834) 2022-06-22 18:29:28 +02:00
Jan Tojnar 347f9a3eda
[contents] Add MIME type for mp3 (#2809)
Without this, format tests fail on systems without `/etc/mime.types`.
2022-06-09 22:41:10 -04:00
Dag cb05cacd6a
fix: add 429 to status codes (#2757) 2022-05-27 15:25:12 +02:00
Dag 5cc34b884a
[core] Improve getContents docs (#2742) 2022-05-22 21:27:23 -04:00
Dag 7d00b0c5df
fix: include http code in exception (#2726) 2022-05-17 23:47:12 +02:00
Dag 0212c4790f
fix: connectivityaction (#2725) 2022-05-17 23:46:37 +02:00
Dag fcc3707210 refactor: swap the order of sprintf values 2022-05-11 22:37:59 +02:00
Dag d5e9dbf47d refactor: restore some useful comments 2022-05-11 22:35:03 +02:00
Dag f5a51038cc fix: error when passing null values
This bug was introduced by me when refactoring the http client.

Fixes:

Fatal error: Uncaught TypeError: Argument 2 passed to getContents() must be of the type array, null given
2022-05-08 04:42:24 +02:00
Dag 5d77d14f9d
feat: add retry logic to the http client (#2692)
* refactor: extract http client

* feat: add retry logic to http client
2022-05-08 03:58:57 +02:00
Patrick Collins e1c898848f
[contents.php] Fix incorrect reference to UnexpectedResponseException's responseBody. (#2677) 2022-04-23 10:04:56 +05:00
dag c3a106892d
fix: require curl extension (#2617) 2022-04-10 18:54:18 +02:00
dag cce11964a4
feat: add a timeout option for http client (#2600) 2022-04-08 21:22:13 +02:00
dag bed20e9f28
feat: extract curl ua to config value (#2568)
* exclude config.default.ini.php from phpcs
2022-04-03 09:37:39 +02:00
Dag b6d1c7a58f fix: php notice
Fixes:
Notice: Undefined variable: message in /home/rssbridge/public/lib/contents.php on line 39
2022-03-24 13:52:02 +05:00
Dag 384790537b fix: bug in cloudflare response detection
The cloudflare server header was not recognized in
some cases such as when the server header is "server"
or when the header value is "Cloudflare".
2022-03-24 13:52:02 +05:00
Dag 7bdc53125c fix: properly verify the existence of the curl module
This fixes a bug where it didnt use curl from cli
even though it's installed.

I believe this preserves the original intention to
not require the curl module to be installed.

https://github.com/RSS-Bridge/rss-bridge/pull/979
2022-03-24 13:52:02 +05:00
Paul Staroch 18504f2356
[lib/contents.php] Use variable name 'retVal' instead of 'retval' as variable names are case-sensitive (#2498) 2022-03-14 05:46:30 +05:00
Eugene Molotov 05273a9278
[core] Make getContents exceptions to be handled correctly and correct exception message (#2447)
This commit fixes following issues:
1. 'Unexpected response' error message was returned, even if upstream did not return anything
2. Inability to handle non-20x messages with checking response body
2022-03-12 01:18:01 +05:00
Martin Leyrer 3764348b76
[core] Accept additional "successful" 2xx status codes (#2310) 2021-10-30 01:38:25 +05:00
Thomas 771b851b52
[contents.php] Fix logical error in getSimpleHTMLDOMCached function (#1974)
Previously content was only loaded from cache when debug mode was enabled (the opposite of the expected behavior)
2021-02-09 17:40:16 +05:00
Eugene Molotov 3226a5e31e [core] Use more correct text to indicate errored response from upstream 2020-12-23 17:49:11 +05:00
Joshua Coales d05a8b79fe
[contents.php] Fix return type hints (#1824) 2020-11-08 12:19:18 +05:00
triatic 3ad126cdf2
[core] Add headers to file_get_contents (#1623)
Add response headers to file_get_contents() method. Headers are used by some bridges.
2020-06-25 12:22:05 +02:00
logmanoriginal 3bc8c9468a phpcs: Always use long array syntax
Most of the code in RSS-Bridge uses the long array syntax.
This commit adds a check to enforce using this syntax over
the short array syntax.

All failures have been fixed.
2019-11-01 18:06:55 +01:00
LogMANOriginal cdc1d9c9ba
action: Add action to check bridge connectivity (#1147)
* action: Add action to check bridge connectivity

It is currently not simply possible to check if the remote
server for a bridge is reachable or not, which means some
of the bridges might no longer work because the server is
no longer on the internet.

In order to find those bridges we can either check each
bridge individually (which takes a lot of effort), or use
an automated script to do this for us.

If a server is no longer reachable it could mean that it is
temporarily unavailable, or shutdown permanently. The results
of this script will at least help identifying such servers.

* [Connectivity] Use Bootstrap container to properly display contents

* [Connectivity] Limit connectivity checks to debug mode

Connectivity checks take a long time to execute and can require a lot
of bandwidth. Therefore, administrators should be able to determine
when and who is able to utilize this action. The best way to prevent
regular users from accessing this action is by making it available in
debug mode only (public servers should never run in debug mode anyway).

* [Connectivity] Split implemenation into multiple files

* [Connectivity] Make web page responsive to user input

* [Connectivity] Make status message sticky

* [Connectivity] Add icon to the status message

* [contents] Add the ability for getContents to return header information

* [Connectivity] Add header information to the reply Json data

* [Connectivity] Add new status (blue) for redirected sites

Also adds titles to status icons (Successful, Redirected, Inactive, Failed)

* [Connectivity] Fix show doesn't work for inactive bridges

* [Connectivity] Fix typo

* [Connectivity] Catch errors in promise chains

* [Connectivity] Allow search by status and update dynamically

* [Connectivity] Add a progress bar

* [Connectivity] Use bridge factory

* [Connectivity] Import Bootstrap v4.3.1 CSS
2019-10-31 22:02:38 +01:00
triatic b6be18d585 [contents] Respect passed headers for file_get_contents() (#1234)
* [contents] Respect passed headers for file_get_contents()
2019-07-29 12:05:13 +02:00
logmanoriginal 2460b67886 cache: Refactor cache factory to non-static class
The cache factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.

References #1001
2019-06-18 19:04:19 +02:00
logmanoriginal 70542686bb [contents] Fix parsing of incomplete headers
Response headers may contain fields with no values.

Example:
  "Referrer-Policy: "

In this case the current implementation of explode() results in an
error because there is no content after ": ". Changing the delimiter
to ":" and trimming the value manually fixes that issue.
2019-06-09 17:18:08 +02:00
fulmeek 21d3bf3b60 caches: Refactor the API (#1060)
- For consistency, functions should always return null on non-existing data.

- WordPressPluginUpdateBridge appears to have used its own cache instance in the past. Obviously not used anymore.

- Since $key can be anything, the cache implementation must ensure to assign the related data reliably; most commonly by serializing and hashing the key in an appropriate way.

- Even though the default path for storage is perfectly fine, some people may want to use a different location. This is an example how a cache implementation is responsible for its requirements.
2019-04-29 20:12:43 +02:00
logmanoriginal 88aae6fd95 core: Apply changes to fix broken Travis builds
Travis-CI recently got updated, which causes existing builds to fail.
For example: https://travis-ci.org/RSS-Bridge/rss-bridge/builds/507568117

Indenting multi-line arguments of functions fixes it.
2019-03-20 19:23:22 +01:00
logmanoriginal 556a417dd6 core: Add support for custom cache types via config.ini.php
This commit adds support for a new parameter which specifies the type
of cache to use for caching. It is specified in config.ini.php:

 [cache]

 type = "..."

Currently only one type of cache is supported (see /caches). All uses
of 'FileCache' were replaced by this configuration option.

Note: Caching currently depends on files and folders (due to FileCache).
Experience may vary depending on the selected cache type. For now always
check if FileCache is working before testing alternative types.

References #1000
2019-02-06 18:52:44 +01:00
triatic 245af35a60 [contents] improve file_get_contents() reporting (#986)
Suppress any errors from file_get_contents() and include the PHP error in the feed instead.
2019-01-06 20:30:02 +01:00
triatic 4095cad9b4 lib: Make cURL module requirement optional (#979)
When running in CLI mode without certificates, do not require curl module to be loaded.
2018-12-26 22:31:30 +01:00