for chinese

.gitignore

@@ -10,6 +10,7 @@ sphinx.conf
 *.p2p
 downloads.json
 node.exe
+.idea
 
 /dist
 /temp

README.md

@@ -30,6 +30,7 @@ BitTorrent search program for desktop and web. Collect and navigate over base of
 * Drag and drop torrents (expand local search database with specific torrents)
 * Descriptions association from trackers
 * Torrent generation and automatic adding to search DB
+* [WebSockets & REST API for server/search engine. You can made search request and create your own UI client.](docs/API.md)
 
 ## Architecture
 ![Basic Architecture](docs/img/ratsarch.png)
@@ -91,6 +92,8 @@ Now you can get access to web interface on 8095 port: http://localhost:8095
 
 [More about server compatibility and known issues](docs/SERVER_COMPATIBILITY.md)
 
+[API usage implementation for clients](docs/API.md)
+
 ## Docker image
 
 You can simply run docker image with prepared server version. Just download last sources:

app/donate.html

@@ -63,21 +63,28 @@
                     C243.779,80.572,238.768,71.728,220.195,71.427z"/>
   <center>
 
-  <div class='patreon' style="display: flex;">
+  <div class='patreon' style="display: flex; flex-wrap: wrap; justify-content: center;">
-  <div class="widget">
+    <div class="widget">
-  <a href="https://opencollective.com/RatsSearch">
+    <a href="https://github.com/sponsors/DEgITx">
 
-  Subscribe on Open Collective (continuous support development with 1$ or more)
+    Support development directly via GitHub sponsorship  (recomended)
-  <img src="open-collective.png" style="width: 400px; padding-top: 12px; opacity: 0.9;" />
+    <img src="github.jpg" style="width: 400px; padding-top: 12px; opacity: 0.9;" />
 
-  </a>
+    </a>
-  </div>
+    </div>
-  <iframe src="https://money.yandex.ru/quickpay/shop-widget?writer=seller&targets=Rats%20Search&targets-hint=&default-sum=500&button-text=11&payment-type-choice=on&mobile-payment-type-choice=on&hint=&successURL=&quickpay=shop&account=410012059502693" width="450" height="213" frameborder="0" allowtransparency="true" scrolling="no"></iframe>
+    <div class="widget">
+    <a href="https://opencollective.com/RatsSearch">
+
+    Subscribe on Open Collective (continuous support development with 1$ or more)
+    <img src="open-collective.png" style="width: 400px; padding-top: 12px; opacity: 0.9;" />
+
+    </a>
+    </div>
   </div>
 
   <br />
-  <div style='display: flex; align-items: center; width: 100%; justify-content: center;'><img style="height: 25px; width: 25px; padding: 5px;" src="https://png.icons8.com/ios/1600/webmoney.png" /> WMR (Webmoney):&nbsp; <b>R227938595852</b></div>
+  <!-- <div style='display: flex; align-items: center; width: 100%; justify-content: center;'><img style="height: 25px; width: 25px; padding: 5px;" src="https://png.icons8.com/ios/1600/webmoney.png" /> WMR (Webmoney):&nbsp; <b>R227938595852</b></div>
-  <div style='display: flex; align-items: center; width: 100%; justify-content: center;'><img style="height: 25px; width: 25px; padding: 5px;" src="https://png.icons8.com/ios/1600/webmoney.png" /> WMZ (Webmoney):&nbsp; <b>Z133588309220</b></div>
+  <div style='display: flex; align-items: center; width: 100%; justify-content: center;'><img style="height: 25px; width: 25px; padding: 5px;" src="https://png.icons8.com/ios/1600/webmoney.png" /> WMZ (Webmoney):&nbsp; <b>Z133588309220</b></div> -->
   </center>
 </body>
 </html>

docs/API.md

@@ -0,0 +1,125 @@
+# Basic description
+
+Rats Search server using WebSockets for internal communication. It require two way communication and using socket.io internally. As alternative added REST API implementation, but because of one-way limitation, it require periodic check of backward messages with **/api/queue** . You free to choise one of the ways for communication and implementation of your own client.
+
+## WebSockets communication (1-way)
+
+
+## REST API communication (2-way)
+
+### Installation
+
+At first you need to prepare server non-interface version
+
+```bash
+git clone --recurse-submodules https://github.com/DEgITx/rats-search.git
+npm install --force
+npm run buildweb
+npm run server
+```
+
+### Configuration
+
+You need to enable REST API configuration (disabled by default):
+
+edit **rats.json** (change only restApi value):
+```json
+{
+    "restApi": true
+}
+```
+
+set restApi to true, save
+
+### API usage
+
+#### Search of torrents
+
+endpoint (GET REQUEST):
+```
+https://localhost:8095/api/searchTorrent?text=DEgITx&navigation={}
+```
+
+example of request:
+```json
+{
+    "text": "Search Name of the torrent",
+    "navigation": {
+        "index": 0,
+        "limit": 10,
+        "orderBy": "order_field",
+        "orderDesc": "DESC",
+        "safeSearch": false
+    }
+}
+```
+
+| Field | Type | Optional | Default Value | Description |
+| ----- | ---- | -------- | ------------- | ----------- |
+| text | string | ❎ |  | torrent search name |
+| navigation | object (Navigation) | ✅ |  | object with navigation params |
+|       index | int | ✅ | 0 | stating of torrent index of navigation |
+|       limit | int | ✅ | 10 | max number of results on page |
+|       orderBy | text | ✅ |  | field which is using for order results |
+|       orderDesc | enum [**DESC, ASC**] | ✅ | ASC | sort direction of the field |
+|       safeSearch | bool | ✅ | true | disable/enable safe search for torrents |
+|       type | string | ✅ |  | type of content for search |
+|       size | object (Interval) | ✅ |  | size of torrent |
+|            min | uint64 | ✅ |  | minumum size of the torrent |
+|            max | uint64 | ✅ |  | maximum size of the torrent |
+|       files | object (Interval) | ✅ |  | files on the torrent |
+|            min | int | ✅ |  | minumum size of the torrent |
+|            max | int | ✅ |  | maximum size of the torrent |
+
+### Reading queue
+
+As said before after each request and periodicly you need to read queue for additional messages.
+
+endpoint (GET REQUEST):
+```
+https://localhost:8095/api/queue
+```
+
+after executing of search **/api/searchTorrent** request **additional result of search will be in queue**!
+
+### Search of the torrent by files
+
+endpoint (GET REQUEST):
+```
+https://localhost:8095/api/searchFiles?text=TorrentWithFileName&navigation={}
+```
+
+| Field | Type | Optional | Default Value | Description |
+| ----- | ---- | -------- | ------------- | ----------- |
+| text | string | ❎ |  | torrent search name |
+| navigation | object (Navigation) | ✅ |  | object with navigation params |
+|       index | int | ✅ | 0 | stating of torrent index of navigation |
+|       limit | int | ✅ | 10 | max number of results on page |
+|       orderBy | text | ✅ |  | field which is using for order results |
+|       orderDesc | enum [**DESC, ASC**] | ✅ | ASC | sort direction of the field |
+|       safeSearch | bool | ✅ | true | disable/enable safe search for torrents |
+
+### Recheck trackers info for the torrent
+
+endpoint (GET REQUEST):
+```
+https://localhost:8095/api/checkTrackers?hash=29ebe63feb8be91b6dcff02bacc562d9a99ea864
+```
+
+| Field | Type | Optional | Default Value | Description |
+| ----- | ---- | -------- | ------------- | ----------- |
+| hash | string | ❎ |  | torrent hash to refresh token |
+
+### Top torrents
+
+endpoint (GET REQUEST):
+```
+https://localhost:8095/api/topTorrents?type=video&navigation={"time":"week"}
+```
+
+| Field | Type | Optional | Default Value | Description |
+| ----- | ---- | -------- | ------------- | ----------- |
+| type | string | ❎ |  | type of category for top of the torrents |
+| navigation | object (Navigation) | ✅ |  | object with navigation params (check /api/searchTorrent for mo details) |
+|       time | enum [hours, week, month] | ✅ |  | time for the top
+

package.json

@@ -109,7 +109,7 @@
     "buildweb": "node src/background/webpack.js"
   },
   "dependencies": {
-    "@electron/remote": "^2.0.8",
+    "@electron/remote": "^2.0.11",
     "ansi-256-colors": "^1.1.0",
     "bencode": "2.0.1",
     "bitfield": "3.0.0",
@@ -120,7 +120,7 @@
     "detect-onebyte-encoding": "^1.0.3",
     "electron-context-menu": "^3.3.0",
     "electron-log": "^4.4.8",
-    "electron-updater": "^5.3.0",
+    "electron-updater": "^6.1.1",
     "fs-jetpack": "^4.3.1",
     "glob": "^7.2.0",
     "google": "^2.1.0",
@@ -162,8 +162,8 @@
     "chai": "^4.3.6",
     "copy-webpack-plugin": "^11.0.0",
     "css-loader": "^6.7.1",
-    "electron": "24.3.1",
+    "electron": "26.1.0",
-    "electron-builder": "23.6.0",
+    "electron-builder": "24.6.3",
     "eslint": "^8.17.0",
     "eslint-plugin-react": "^7.30.0",
     "express": "^4.18.1",

src/app/config-page.js

@@ -216,6 +216,16 @@ export default class ConfigPage extends Page {
 								}}
 							/>
 
+							<Toggle
+								style={{marginTop: '10px'}}
+								label={__('REST API support')}
+								toggled={this.options.restApi}
+								onToggle={(e, checked) => {
+									this.options.restApi = checked
+									this.forceUpdate()
+								}}
+							/>
+
 						</div>
 					</Tab>
 					<Tab value='p2p' label={__("P2P settings")} icon={<SvgIcon viewBox="0 0 47 47">

src/background/background.js

@@ -102,6 +102,11 @@ if(portative)
 process.on('unhandledRejection', r => logTE('system', 'Rejection:', r));
 process.on('uncaughtException', (err, origin) => logTE('system', 'Exception:', err, 'Origin:', origin));
 
+if (env.name !== "production" && (!fs.existsSync(__dirname + '/../imports') || fs.readdirSync(__dirname + '/../imports').length == 0)) {
+	logTE('system', 'You are not clonned submodules correctly, please use git clone --recurse-submodules https://github.com/DEgITx/rats-search.git');
+	process.exit(1);
+}
+
 const gotTheLock = app.requestSingleInstanceLock()
 if (!gotTheLock) {
 	logT('app', 'closed because of second application')

src/background/server.js

@@ -40,6 +40,11 @@ if(majorVersion < 8)
 	process.exit(1);
 }
 
+if (!fs.existsSync(__dirname + '/../../imports') || fs.readdirSync(__dirname + '/../../imports').length == 0) {
+	logTE('system', 'You are not clonned submodules correctly, please use git clone --recurse-submodules https://github.com/DEgITx/rats-search.git');
+	process.exit(1);
+}
+
 app.use(express.static('web'));
 
 appConfig.restApi = true;
@@ -58,11 +63,32 @@ io.on('connection', (socket) =>
 	}
 })
 
+let responceRestQueue = [];
+
+if (appConfig.restApi) {
+	app.get('/api/queue', (req, res) => {
+		const uniqueId = Math.random().toString(16).slice(2) + '_' + (new Date()).getTime();
+		logT('rest', 'queue responce', uniqueId, 'size', responceRestQueue.length);
+		res.send({id: uniqueId, queue: responceRestQueue})
+		// clear queue after the read of json queue
+		responceRestQueue = [];
+	});
+}
+
 const start = async () => 
 {
 	({ sphinx } = await startSphinx(() => {
 		dbPatcher(() => {
-			spider = new spiderCall((...data) => io.sockets.emit(...data), (message, callback) => {
+			spider = new spiderCall((...data) => { 
+				if (appConfig.restApi) {
+					if (responceRestQueue.length < 1000) {
+						responceRestQueue.push(data);
+					} else {
+						logTE('rest', 'max 1000 queue records, please use /api/queue to clean records')
+					}
+				}
+				return io.sockets.emit(...data) 
+			}, (message, callback) => {
 				socketMessages[message] = callback
 				if (appConfig.restApi) {
 					app.get('/api/' + message, (req, res) => {

src/background/sphinx.js

@@ -37,10 +37,13 @@ const writeSphinxConfig = async (rootPath, dbPath, params = {}) => {
   {
     type = rt
     path = ${dbPath}/database/torrents
-    
+
     min_prefix_len = 3
     expand_keywords = 1
-    
+    charset_type = utf-8
+    charset_table = 0..9, A..Z->a..z, a..z, U+4E00..U+9FFF->U+4E00..U+9FFF, U+3400..U+4DBF->U+3400..U+4DBF, U+20000..U+2A6DF->U+20000..U+2A6DF, U+2A700..U+2B73F->U+2A700..U+2B73F, U+2B740..U+2B81F->U+2B740..U+2B81F, U+2B820..U+2CEAF->U+2B820..U+2CEAF, U+2CEB0..U+2EBEF->U+2CEB0..U+2EBEF
+
+
     rt_attr_string = hash
     rt_attr_string = name
     rt_field = nameIndex
@@ -59,7 +62,7 @@ const writeSphinxConfig = async (rootPath, dbPath, params = {}) => {
     rt_attr_uint = good
     rt_attr_uint = bad
     rt_attr_json = info
-    
+
 	stored_only_fields = ipv4
   }
 
@@ -67,7 +70,11 @@ const writeSphinxConfig = async (rootPath, dbPath, params = {}) => {
   {
     type = rt
     path = ${dbPath}/database/files
-      
+
+    charset_type = utf-8
+    charset_table = 0..9, A..Z->a..z, a..z, U+4E00..U+9FFF->U+4E00..U+9FFF, U+3400..U+4DBF->U+3400..U+4DBF, U+20000..U+2A6DF->U+20000..U+2A6DF, U+2A700..U+2B73F->U+2A700..U+2B73F, U+2B740..U+2B81F->U+2B740..U+2B81F, U+2B820..U+2CEAF->U+2B820..U+2CEAF, U+2CEB0..U+2EBEF->U+2CEB0..U+2EBEF
+
+
     rt_field = path
     rt_attr_string = hash
     rt_field = size
@@ -80,7 +87,7 @@ const writeSphinxConfig = async (rootPath, dbPath, params = {}) => {
   {
       type = rt
       path = ${dbPath}/database/version
-      
+
       rt_attr_uint = version
       rt_field = versionIndex
   }
@@ -89,7 +96,7 @@ const writeSphinxConfig = async (rootPath, dbPath, params = {}) => {
   {
       type = rt
       path = ${dbPath}/database/store
-      
+
       rt_field = storeIndex
       rt_attr_json = data
       rt_attr_string = hash
@@ -269,7 +276,7 @@ module.exports = async (callback, dataDirectory, onClose, params = {}) => {
 				sphinx.version = manticoreVersion[1];
 				logT('sphinx', 'sphinx version', sphinx.version);
 			}
-	
+
 			if(windowsEncodingFix && data.includes('failed to parse config file'))
 			{
 				logT('sphinx', 'encoding rewrite failed, forcing restart of application to fix that problem')
@@ -305,12 +312,12 @@ module.exports = async (callback, dataDirectory, onClose, params = {}) => {
 				sphinx.onClose = onFinish
 			if(replaceFinish)
 				sphinx.replaceOnClose = true // sometime we don't want to call default callback
-            
+
-			if (!sphinx.isExternal) 
+			if (!sphinx.isExternal)
 			{
 				logT('sphinx', `stoping with sphinx stopwait`);
 				exec(`"${sphinxPath}" --config "${config}" --stopwait`)
-			} 
+			}
 			else
 			{
 				logT('sphinx', `ignoring sphinx closing because external sphinx instance`)
@@ -433,7 +440,7 @@ module.exports = async (callback, dataDirectory, onClose, params = {}) => {
 
 			const checkNullFile = (file) => new Promise((resolve) => {
 				let f = fs.createReadStream(file)
-				f.on('data', (chunk) => {  
+				f.on('data', (chunk) => {
 					for(const byte of chunk)
 						if(byte != 0)
 						{
@@ -450,7 +457,7 @@ module.exports = async (callback, dataDirectory, onClose, params = {}) => {
 			const probablyCoruptedFiles = await findFiles(`${sphinx.directoryPath}/**/*.+(meta|ram)`)
 			let brokenFiles = await Promise.all(probablyCoruptedFiles.map(file => checkNullFile(file)))
 			brokenFiles = probablyCoruptedFiles.filter((file, index) => !brokenFiles[index])
-            
+
 			brokenFiles.forEach(file => {
 				logT('sphinx', 'FIXDB: clean file because of broken', file)
 				fs.unlinkSync(file)
@@ -467,4 +474,4 @@ module.exports = async (callback, dataDirectory, onClose, params = {}) => {
 	}
 
 	return await start(callback)
-}
+}

translations/cn.json

@@ -198,6 +198,7 @@
         "File": "文件",
         "Folder": "文件夾",
         "Generate": "產生",
-        "Dark mode theme": "Dark mode theme"
+        "Dark mode theme": "Dark mode theme",
+        "REST API support": "REST API support"
     }
 }

translations/en.json

@@ -198,6 +198,7 @@
         "File": "File",
         "Folder": "Folder",
         "Generate": "Generate",
-        "Dark mode theme": "Dark mode theme"
+        "Dark mode theme": "Dark mode theme",
+        "REST API support": "REST API support"
     }
 }

translations/ru.json

@@ -198,6 +198,7 @@
         "File": "Файл",
         "Folder": "Папка",
         "Generate": "Сгенерировать",
-        "Dark mode theme": "Темная тема"
+        "Dark mode theme": "Темная тема",
+        "REST API support": "REST API support"
     }
 }

translations/ua.json

@@ -198,6 +198,7 @@
         "File": "Файл",
         "Folder": "Папка",
         "Generate": "Згенерувати",
-        "Dark mode theme": "Dark mode theme"
+        "Dark mode theme": "Dark mode theme",
+        "REST API support": "REST API support"
     }
 }