You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Porter is the all-purpose PHP data importer. She fetches data from anywhere and serves it as a single record or an iterable [record collection](#record-collections), encouraging processing one record at a time instead of loading full data sets into memory at once. Her [durability](#durability) feature provides automatic, transparent recovery from intermittent network connectivity errors by default.
14
13
15
-
Porter's [interface trichotomy](#overview) of [providers](#providers), [resources](#resources) and [connectors](#connectors) maps well to APIs. For example, a typical API such as GitHub would define the provider as GitHub, a resource as `GetUser` or `ListRepositories` and the connector could be [HttpConnector][].
14
+
Porter's [interface triad](#overview) of [providers](#providers), [resources](#resources) and [connectors](#connectors) maps well to APIs. For example, a typical API such as GitHub would define the provider as GitHub, a resource as `GetUser` or `ListRepositories` and the connector could be [HttpConnector][].
16
15
17
16
Porter provides a dual API for synchronous and [asynchronous](#asynchronous) imports, both of which are concurrency safe, so multiple imports can be paused and resumed simultaneously. Asynchronous mode allows large scale imports across multiple connections to work at maximum efficiency without waiting for each network call to complete.
18
17
@@ -26,32 +25,32 @@ Contents
26
25
--------
27
26
28
27
1.[Benefits](#benefits)
29
-
1.[Quick start](#quick-start)
30
-
1.[About this manual](#about-this-manual)
31
-
1.[Usage](#usage)
32
-
1.[Porter's API](#porters-api)
33
-
1.[Overview](#overview)
34
-
1.[Import specifications](#import-specifications)
35
-
1.[Record collections](#record-collections)
36
-
1.[Asynchronous](#asynchronous)
37
-
1.[Transformers](#transformers)
38
-
1.[Filtering](#filtering)
39
-
1.[Durability](#durability)
40
-
1.[Caching](#caching)
41
-
1.[Architecture](#architecture)
42
-
1.[Providers](#providers)
43
-
1.[Resources](#resources)
44
-
1.[Connectors](#connectors)
45
-
1.[Requirements](#requirements)
46
-
1.[Limitations](#limitations)
47
-
1.[Testing](#testing)
48
-
1.[Contributing](#contributing)
49
-
1.[License](#license)
28
+
2.[Quick start](#quick-start)
29
+
3.[About this manual](#about-this-manual)
30
+
4.[Usage](#usage)
31
+
5.[Porter's API](#porters-api)
32
+
6.[Overview](#overview)
33
+
7.[Import specifications](#import-specifications)
34
+
8.[Record collections](#record-collections)
35
+
9.[Asynchronous](#asynchronous)
36
+
10.[Transformers](#transformers)
37
+
11.[Filtering](#filtering)
38
+
12.[Durability](#durability)
39
+
13.[Caching](#caching)
40
+
14.[Architecture](#architecture)
41
+
15.[Providers](#providers)
42
+
16.[Resources](#resources)
43
+
17.[Connectors](#connectors)
44
+
18.[Requirements](#requirements)
45
+
19.[Limitations](#limitations)
46
+
20.[Testing](#testing)
47
+
21.[Contributing](#contributing)
48
+
22.[License](#license)
50
49
51
50
Benefits
52
51
--------
53
52
54
-
* Defines an **interface trichotomy** for data imports: [providers](#providers) represent one or more [resources](#resources) that fetch data from [connectors](#connectors). These interfaces make it very easy to **test and mock** specific parts of the import lifecycle using industry standard tools, whether we want to mock at the connector level and feed in raw responses or mock at the resource level and feed in hydrated objects.
53
+
* Defines an **interface triad** for data imports: [providers](#providers) represent one or more [resources](#resources) that fetch data from [connectors](#connectors). These interfaces make it very easy to **test and mock** specific parts of the import lifecycle using industry standard tools, whether we want to mock at the connector level and feed in raw responses or mock at the resource level and feed in hydrated objects.
55
54
* Provides **memory-efficient data processing** interfaces that handle large data sets one record at a time, via iterators, which can be implemented using deferred execution with generators.
56
55
*[Asynchronous](#asynchronous) imports offer highly efficient **CPU-bound data processing** for large scale imports across multiple connections concurrently, eliminating network latency performance bottlenecks. Concurrency can be **rate-limited** using [throttling](#throttling).
57
56
* Protects against intermittent network failures with [durability](#durability) features that transparently and **automatically retry failed data fetches**.
@@ -479,21 +478,12 @@ It is important to define a canonical order for hashed inputs such that identica
479
478
480
479
To support Porter's durability features a connector may throw a subclass of `RecoverableConnectorException` to signal that the fetch operation can be retried. Execution will halt as normal if any other exception type is thrown. It is recommended to throw a recoverable exception type when the fetch operation is idempotent.
481
480
482
-
Requirements
483
-
------------
484
-
485
-
-[PHP 7.1](http://php.net/)
486
-
-[Composer](https://getcomposer.org/)
487
-
488
481
Limitations
489
482
-----------
490
483
491
484
Current limitations that may affect some users and should be addressed in the near future.
492
485
493
486
- No end-to-end data steaming interface.
494
-
- Caching does not support asynchronous imports.
495
-
-[Sub-imports][] do not support async.
496
-
- No import rate throttle for synchronous imports.
0 commit comments